pm_EnsembleSpaces.tex

\def\>{\rangle}
\def\<{\langle}

%\DeclareMathOperator{\mix}{mix}
%\DeclareMathOperator{\component}{comp}
%\DeclareMathOperator{\cospan}{cospan}
\newcommand\mix{\mathrm{mix}}
\newcommand\component{\mathrm{comp}}
\newcommand\cospan{\mathrm{cospan}}
\newcommand\dist{\mathrm{dist}}
\newcommand\hull{\mathrm{hull}}
\newcommand\shull{\mathrm{shull}}
\newcommand\chull{\mathrm{chull}}
\newcommand\support{\mathrm{supp}}
\newcommand\scap{\mathrm{scap}}
\newcommand\fraction{\mathrm{frac}}
\newcommand\fcap{\mathrm{fcap}}

\newcommand\vspan{\mathrm{span}}
\newcommand\cl{\mathrm{cl}}

\def\separate{\downmodels}
\def\nseparate{\ndownmodels}
\def\ortho{\perp}
\def\northo{\nperp}

%\NewDocumentCommand{\separate}{o}{
%	\downmodels
%	\IfNoValueTF{#1}{}{_{#1}}%
%}
%
%\NewDocumentCommand{\nseparate}{o}{
%	\ndownmodels
%	\IfNoValueTF{#1}{}{_{#1}}%
%}

\newcommand{\ens}[1][e] {\mathsf{#1}} % Ensemble
\newcommand{\Ens}[1][E] {\mathcal{#1}} % Ensemble space

\newcommand{\titledbreak}[1]{
	\newpage
	\vspace{3em}  % Add vertical space before the title
	\noindent
	\centerline{\huge\centering\textbf{#1}} % Print the titled break title in bold
	\par\nobreak
%	\vspace{0.5em}  % Add some vertical space after the title
	\noindent\rule{\textwidth}{0.5pt}  % Add a horizontal rule
%	\vspace{1.5em}  % Add vertical space after the rule
}


\colorlet{ensembleFill}{black!10!white}

\newenvironment{openproblem}[3]{\section{Open problem: #1}\label{#2} \emph{Tags: #3} \newline \newline}{}

\chapter{Ensemble spaces}

In this chapter we aim to develop a general theory of states and processes that is applicable to any physical system. The core concept is that of an ensemble as we have found that ensembles in both classical and quantum mechanics have a very similar structure that can be abstracted and generalized. The goal is to find necessary requirements for ensembles that can serve as basic axioms and then further suitable assumptions to recover the different theories (e.g. classical, quantum or thermodynamics).

The basic premise is that physical theories are primarily about ensembles. At a practical level, most of the time we can only prepare and measure statistical properties as we do not have perfect control over any system (i.e. all measurements are really statistical). The cases where properties can be prepared with one hundred percent reliability can still be understood as ensembles of identical preparations. At a conceptual level, the goal of physics is to write laws that can be repeatedly tested: every time that one prepares a system according to a particular procedure and lets it evolve in particular conditions, he will obtain a particular result. That is, the idea of repeatability of experimental results implicitly assumes that the objects of scientific inquiry are not single instances, but the infinite collections of all reproducible instances. This means that any physical theory, at the very least, will have to provide a mathematical representation for its ensembles.

By ensemble we mean what is usually meant in statistical mechanics: we have a preparation device that follows some known recipe; its output is varied but it is consistently varied (i.e. its statistical properties are well defined); the collection of all possible outputs taken as one object is an ensemble. In classical physics, ensembles are probability distributions over the full description of the system, over classical phase space. In quantum mechanics, ensembles are represented by density matrices and density operators. In the standard approach statistical ensembles are defined on top of the space of ``true'' physical states (e.g. microstates, pure states, ...). We will proceed in the opposite way: we will start from the ensembles and recover the states as the ``most pure'' ensembles. There are two main advantages: the first is that we can create a theory that is agnostic about what the fundamental states are, and is therefore general. The second is that this approach is more in line with experimental practice: the experimental data is about statistical ensembles only and the pure states are idealizations that are useful as a mental model or for calculation.

At this point, we have identified three main requirements for ensembles. First, they must be experimentally well defined. This means that there need to be enough experimentally verifiable statements to fully characterize them. This will impose a topology on the ensemble space. Second, we can always perform statistical mixtures: given two ensembles, we can create a third one by selecting the first or the second according to a certain probability. This will impose a convex structure on the ensemble space. Third, ensembles will need a well-defined entropy which quantifies the variability of the elements within the ensemble. Since the variability cannot decrease when performing a statistical mixture and can only increase up to the variability introduced by the selection, the entropy will have to satisfy certain bounds. From these axioms, many general results can be proven. An ensemble space will be a convex subset of a vector space that will extend over a bounded interval in each direction. The concavity of the entropy will impose a metric over the space, turning it into a geometric space. Each ensemble can be characterized by a subadditive measure, which becomes a probability measure in the classical case. This makes the space of physical theories a lot more constrained than one may imagine at first.

Many problem are still open, as they touch unsettled mathematical questions, particularly in the infinite dimensional case. Therefore this chapter will include conjectures that may or may not be true, which the reader is encouraged to try proving (or disproving).

Note: all $\log$s are assumed to be in base 2.

\section{Review of standard cases}

We will start this chapter by reviewing three cases: discrete classical ensembles, continuous classical ensembles and quantum ensembles. These will be useful both to form an intuition for ensembles and to serve as targets that the whole theory needs to reproduce. We will also go though a series of problematic details and exceptions that we will need to address in the development of the general theory.

\subsection{Discrete classical ensemble spaces}

\begin{defn}
	A \textbf{discrete classical ensemble space} is the space of probability distributions over a countable sample space equipped with the Shannon entropy. That is, a discrete classical ensemble space $\Ens$ is the space of probability distributions over a discrete set $X$ of countably many elements. Each element can thus be identified by a sequence $p_i$ such that $\sum_i p_i = 1$, where $p_i$ is the probability associated to each element $x_i \in X$. The entropy is given by $S(\ens) = - \sum_i p_i \log p_i$.
\end{defn}

\subsubsection{Finite case}

The space of classical distributions over a discrete space corresponds to a \href{https://en.wikipedia.org/wiki/Simplex}{simplex}. In the finite case, the pure states $X = \{x_i\}_{i=1}^{n} \subset \Ens$ are finitely many and each ensemble $\ens = \sum_i p_i x_i$ is uniquely identified by a decomposition of pure states. Effectively, each ensemble is a probability distribution over the pure states. Mathematically, each point of the space is a convex combination of the vertices. The simplex has a center point, which corresponds to the maximally mixed state, a uniform distribution over all pure states. 

The entropy is given by the Shannon entropy $-\sum_i p_i \log p_i$. This means that the entropy of each pure state is zero and the entropy of the maximally mixed state is $\log n$ where $n$ is the number of pure states. The entropy increases as we go from pure states to the maximally mixed state. The level sets (i.e. the fibers) of the entropy form a series of concentric ``shells'' that foliates the space.

Note that imposing zero entropy on all pure states is a restrictive condition that does not apply in general. To see this, consider the case where the state is defined by the number of molecules for two substances. This space is the product of two independent variables $n_a$ and $n_b$. If we have a uniform distribution over $N_a$ cases of $n_a$ and $N_b$ cases of $n_b$, the total number of cases is $N_a N_b$. Therefore the entropy of the joint state is the sum of the entropy of the marginals. However, if we pair $n_a$ with the total number of molecules $n_{(a+b)}$ we have a problem. The issue is that the variable $n_{(a+b)}$ corresponds to a variable number of joint cases. Therefore the case where the ensemble space is a simplex but the entropy is not the Shannon entropy (i.e. it is the Shannon entropy plus the contributions of entropy from each vertex) is a physically meaningful case that should be possible in the general theory.

\subsubsection{Countable case}

The countable case is, in some respects, not well defined.

The obvious extension is to include all sequences $\{p_i\} \in [0,1]$ whose sum converges to one (i.e. the space of all probability measures over a countable discrete space). Since we cannot create a uniform distribution over infinitely many cases, there is no center point, there is no barycenter. Effectively, there is a ``hole'' in the middle.\footnote{It may be useful to characterize this ``hole'' and the limit points. There should be at least one limit point for each sequence $\{p_i\}$ whose sum converges to a finite $p < 1$. Intuitively, we can keep that part of the distribution constant while we spread the rest uniformly to all other cases. Each should reach a different limit point.}

However, the space of all probability measures is too large. Note that the entropy is not finite for all $\sum_i p_i =1$ (i.e. infinite convex combinations).\footnote{For details, see \href{https://arxiv.org/pdf/1212.5630.pdf}{J. Stat. Mech. (2013) P04010}.} Given that we want the entropy to exist and be finite for all ensembles, this generalization does not seem physically warranted.\footnote{This generalization is called a   \href{https://ncatlab.org/nlab/show/superconvex+space}{superconvex space} in some literature.}

Also note that expectation values are not guaranteed to be finite either, and requiring a particular observable to be finite further restricts the space. This restriction may be desirable for another reason: a discrete ensemble space has no notion of the ordering of the pure states. Physically, this would mean that the states with 1, 100, or 1 trillion particles are ``equally distant'' (i.e. all infinite permutations are allowed). Requiring the expectation of the number of particles to be finite (e.g. $\sum_i N(i) p_i < \infty$) should effectively encode the infinite ordering in the rate of convergence of the probability distributions (i.e. not all infinite permutations would be allowed).

Another problem is identifying the correct topology for the space. The entropy defines a notion of orthogonality, as we will see later, based on the disjoint support of distributions. This suggests that we need a space with an inner product, therefore we should require the probability distribution to be square integrable. However, the inner product could be defined on the square root of the probability distribution, more in line with the quantum case, which would be well defined as probability is never negative. It is not yet clear which is the correct case.

\subsubsection{No uncountable case}

The uncountably infinite case is not physically relevant, as the space cannot be given a second countable discrete topology. Also note that any set of real numbers whose sum is finite can have only countably many non-zero elements. To understand why, note that there can only be finitely many terms above any particular positive value if their sum is to remain finite. Effectively, the uncountable case would be stitching together infinitely many countable cases.

\subsection{Continuous classical ensemble spaces}

\begin{defn}
	A \textbf{continuous classical ensemble space} is the space of probability distributions over classical phase space equipped with the Shannon/Gibbs entropy. That is, it is the space of probability measures $\Ens$ over a symplectic manifold $X$ that is absolutely continuous with respect to the Liouville measure $\mu$. The entropy is given by the Shannon/Gibbs entropy calculated using the probability density (i.e. the Radon-Nikodym derivative between the probability measure $p$ and the Liouville measure $\mu$). That is, $S(\rho) = - \int_X \rho \log \rho d\mu$ where $\rho = \frac{dp}{d\mu}$.
\end{defn}

In the continuous case, the space of ensembles can be understood as the space of non-negative integrable functions over a symplectic manifold (e.g.  over phase space) that integrate to one. That is, if $X$ is a symplectic manifold, then $\Ens = \{ \rho \in L^1(X) \, | \, \rho(x) \geq 0, \, \int_X \rho(x) d\mu = 1 \} $ where $\mu(U)=\int_U \omega^n$ is the Liouville measure. This is a convex set, whose extreme points would be, in the limit, the Dirac measures (i.e. the probability measure all concentrated at a single phase space point). As we cannot reliably prepare a system at an infinitely precise position and momentum, these distributions are not physical. Also, they would correspond to minus infinite entropy. The Dirac measure, and in general all distributions over a set of measure zero, are excluded because they are not absolutely continuous. The absence of the extreme points is important as, when developing standard constructions in the ensemble space, one cannot rely on the existence of extreme points. In general, this points to a difference between the spectra (i.e. the possible values of a random variable) and pure states (i.e. extreme points), which will be even more pronounced in the quantum case.

The symplectic nature of the manifold is required to assign a frame-invariant density to states and a frame-invariant notion of independence between DOFs, as we saw in the classical mechanics section of reverse physics. The entropy is given by $- \int \rho \log \rho d\mu$ where $\mu$ is the Liouville measure and $\rho$ is the probability density over canonical coordinates. If a different measure is used, or if the coordinates are not canonical, the formula gives the wrong result.\footnote{It may be interesting to study the shell of zero entropy states. For example, it should not be path connected. All uniform distributions with support of the same finite size (in terms of the Liouville measure) will have the same entropy. The region, however, need not be contiguous. Since we cannot continuously transform a single region into two disjoint regions, there will be different distributions at zero entropy that cannot be transformed continuously.}


Similarly to the countable discrete case, the entropy can be infinite and expectation values can be infinite. The added complication is the frame invariance: it would not make sense to have finite expectation for position in one frame but infinite in another. Requiring all functions of position and momentum to have finite expectation restricts the distributions to those with finite support. Requiring all polynomial functions of position and momentum to have finite expectation restricts the distributions to those that decay faster than any polynomial. Note that the expectation of all polynomials of position and momentum are not enough to reconstruct the distribution. Furthermore, derivatives of the distribution over position and momentum are needed to determine how the distribution evolves over time given a time evolution map. This may suggest that the proper space of probability measures does not include all the absolutely continuous ones, but only those for which the probability density is a Schwartz function. These details are still to be understood.

The above consideration would seem to rule out probability measures with a discontinuous probability density. This is somewhat of an unclear point. Originally, we thought the probability density should be continuous as only continuous functions can physically represent experimental relationships. However, the relationship given by the measure is not between points and probability but rather between sets and probability. The probability density is, in a sense, not the prime physical object. The measure is. The probability density, in fact, is not uniquely defined as countably many discontinuities can be added and the measure is not changed. This seems to suggest that what happens at a single point, or over a set of measure zero, is not critical. However, it is unclear how to recover continuity of the space without continuity of the distributions. It could be that it is the space of deterministic and reversible transformations that defines continuity. One added benefit of allowing discontinuous probability densities is that the uniform distribution gives the maximum entropy within the set of probability distribution with support over a finite measure set. This makes entropy maximization considerations a lot easier.

Unlike the discrete classical case, subspaces and dimensionality of subspaces cannot be defined without the entropy. The issue is that we need a measure on the set of pure states, and the convex structure cannot provide it. The entropy does, however, as the supremum of the entropy for all distributions with support $U$ is $\log \mu(U)$. As we will see, the entropy can be used to both identify subspaces and recover the Liouville measure.

\subsection{Quantum ensemble spaces}

\begin{defn}
	A \textbf{quantum ensemble space} is the space given by the density matrices/operators of a Hilbert space equipped with the von Neumann entropy. That is, given a separable Hilbert space $\mathcal{H}$ for a quantum system, the ensemble space $\Ens$ is the space of positive semi-definite self-adjoint operators with trace one $M(\mathcal{H})$. The space of pure states $X$ is given by the projective space $P(\mathcal{H})$. The entropy of an ensemble $\rho \in \Ens$ is given by the von Neumann entropy $S(\rho) = -\tr(\rho \log \rho)$.
\end{defn}

\subsubsection{Finite dimensional case}

The simplest non-trivial case is the qubit, for which the Bloch ball is the space of ensembles $\Ens = M(\mathcal{H})$. The interior of the Bloch ball corresponds to mixtures  while the surface corresponds to the pure states $X = P(\mathcal{H}) = \{ |\psi\> \<\psi| \}_{\psi \in \mathcal{H}}$. In quantum ensemble spaces there is no unique decomposition in terms of pure states. Note that the space is exactly characterized by knowing which different mixtures provide the same ensemble.

Multiple decompositions make the ensemble space behave in a way that is a hybrid between the classical discrete and continuous. Pure states are properly a part of the ensemble space, as in the discrete case, and we can describe each mixture in terms of finitely many pure states. However, the pure states form a continuum, therefore we can also define probability densities over the space, convex integrals. For example, for a single qubit, the maximally mixed state (the center of the ball) can be equally described as the equal mixture of any two opposite states (e.g. spin up and spin down, or spin left and spin right). However, it can also be described as the equal mixture of the whole sphere.

Note that complex projective spaces are symplectic, which is what allows one to define frame invariant densities. The goal is to have one argument applied to the generic definition as to why the space of pure states must be symplectic. Also note that the two dimensional sphere is the only symplectic sphere. By homogeneity, we should be able to argue that the space is symmetric around the maximally mixed state, and is therefore a sphere. The symplectic requirement would select dimension two. Note that real and quaternionic spaces would be excluded by this argument.

The von Neumann entropy for the maximally mixed state is $\log n$ where $n$ is the dimensionality of the Hilbert space. Again we see that the maximum entropy gives us a measure of the size of the space. Note that, to calculate the von Neumann entropy, we are diagonalizing the density matrix $\rho$. This means finding a set of orthogonal pure states $x_i$ such that $\rho_i = \sum_i p_i x_i$ is a convex combination. Note that the convex hull of a set of $n$ orthogonal pure states is an $n$-dimensional simplex whose center is the maximally mixed state. Therefore, we are looking for a simplex that contains $\rho$ and the maximally mixed state. In the two dimensional case, $\rho$ is an interior point of the Bloch ball. Take the line that connects $\rho$ to the center. The two points of the sphere are the extreme points for the decomposition. The distance from the points will be proportional to the probability. Because of this property, the von Neumann entropy is the smallest Shannon entropy among all possible decompositions.


\subsection{Countably infinite dimensional case}

The countably infinite dimensional case presents similar problems as the classical case, and adds others. As in the classical infinite cases, the maximally mixed state (i.e. uniform distribution) is not in the convex space and the entropy is not finite for all infinite convex combinations. As in the classical continuous case, there is the issue of finite expectation of position/momentum in all frames. The problem is compounded by the fact that one cannot require finite expectation for all functions of position and momentum: finite support in position automatically implies infinite support on momentum, since the distribution in momentum is the Fourier transform of that in position.

The Hilbert space for a discrete variable with infinite range (e.g. number of particles) and a continuous variable (e.g. position/momentum) is the same. The first is defined as the space of square-summable complex sequences $l^2$ while the second is the space of square integrable complex functions $L^2$. Given that $L^2$ allows a countable basis, the two are isomorphic. This also means that all spaces with finitely many degrees of freedom are also isomorphic. This makes the problem of infinite expectations even more problematic.

Note that Schwartz spaces have finite expectation for all polynomial functions of position and momentum, given that the momentum operator is the derivative of position. Given that infinite permutations can change the rate of convergence, the Schwartz space has an idea of what is further away from the origin, unlike Hilbert spaces. We will likely want to use Schwartz spaces instead of Hilbert spaces to make the physics and mathematics more consistent.

\section{Axiom of ensemble and topology}

Statistical ensembles will be the cornerstone of our general theory for states and processes. In this section we will see how any physical theory must, at least, define a space of ensembles which define the output of all possible processes considered by the theory. Since a physical theory must allow for experimental verifiability, the ensemble space must be endowed with a $\mathsf{T}_0$ second countable topology.

We saw how the principle of scientific objectivity required science to be universal, non-contradictory and evidence based. If our goal is to find laws that govern the evolution of physical systems, however, this is not sufficient. Scientific laws will be statements of the type \statement{every time we prepare this type of system according to this procedure and let it evolve under these conditions, we will find the system in this configuration after some time.} ``Every time'' implies the principle of scientific reproducibility.
\begin{mathSection}
	\textbf{Principle of scientific reproducibility}.
	Scientific laws describe relationships that can always be experimentally reproduced.
\end{mathSection}

Consider the hypothesis that all life on earth descends from a single common ancestor. This is a scientific hypothesis that may be experimentally falsified, but it is about a single event. As such, it is not a scientific law. The theory of evolution through natural selection, instead, describes what always happens to a population given a set of circumstances, and is therefore a law. As such, it does not describe a particular set of living organisms or traits, it applies to all of them but, as a consequence, to none in particular.

The same applies to the laws of physics. Classical Hamiltonian mechanics or quantum mechanics will apply to certain classes of physical systems, describing the common behavior within each class over all possible instantiations, but none in particular. That is, the law is not describing a particular behavior of a particular system, but the common behavior of the aggregate of all similarly prepared systems at all possible times.

The subject of a physical law, then, is not a single system in a single state, but an ensemble: all possible preparations of equivalent systems prepared according to the same procedure. A general theory of states and processes, then, will be a theory about ensembles as this is the least restrictive requirement needed. Any physical theory will \emph{at least} provide us with a set of ensembles, and the physical laws must be able to describe the evolution of those ensembles.

Given that we are still talking about scientific investigation, ensembles must be experimentally well defined and the principle of scientific objectivity applies. This means that ensembles are the possibilities of an experimental domain, which means points of a $\textsf{T}_0$ second countable topological space where the topology corresponds to the natural one defined by the verifiable statements.

The verifiable statements for the ensemble space are statements about the ensembles themselves, either in terms of statistical quantities (e.g. \statement{the average energy of the particle is $3  \pm 0.5 \, eV$}) or in terms of preparation settings (e.g. \statement{the beam goes through a polarizer oriented vertically within 1 degree}). Probability ranges are also typical verifiable statements on ensembles (e.g. \statement{the coin toss will result in heads between 49 and 51 percent of the cases}). Note that the verifiable statements at the level of the ensemble are different from the verifiable statements at the level of each instance. Saying that a coin is fair, for example, is a statement on the ensemble while saying that the outcome at a particular time was heads is a statement on the instance. The two are unrelated: whether the coin is fair is an independent statement with respect to a particular instance. Therefore the topology of the ensemble space and the topology of the random variables are distinct conceptual and mathematical objects (e.g. the topology of the Bloch ball is the standard topology of $\mathbb{R}^3$ but the topology on the values of spin along a given direction is the discrete topology), and it will be much later that the two will be reconciled.\footnote{In fact, many details are still open.}

It should also be clear that ensembles are theoretical objects, idealizations: an infinite collection of instances cannot be realized. What is realized in a laboratory will be a finite version, with all limitations in terms of precision that go with it. One may ask: how can something so idealized represent physical objects? But this is exactly what we do in all other areas of physics: we talk about spheres, perfect fluids, isolated systems or immovable objects. All of these are idealized objects, and we model the world with such abstractions. Ensembles are useful idealizations precisely because they ignore details that are not relevant for the problem at hand. If we want to write physical laws, in fact, we can only write them on those features that are common to all instances. The ensemble represents exactly those and only those features.

Setting ensembles as a primary notion also solves another conceptual problem. The ensemble is not constructed as a limit of infinite instances, which would pose a number of problems. The ensemble describes the preparation procedure, and therefore the collection of instances is potential and comes before the instances. For example, a fair coin can be understood as the collection of instructions for producing and throwing a fair coin. The fact that a fair coin will produce, in a large trial, about half heads and half tails is a consequence of the type of preparation. This gives automatically an interpretation of probability that is more along the lines of propensity, which is more appropriate to express objective causal relations.

\begin{mathSection}\label{pm-es-axiomEnsemble}
	\begin{axiom}[Axiom of ensemble] 
		The state of a system is given by an \textbf{ensemble}, which represents the collection of all possible outputs of a preparation procedure for a physical system. The set of all possible ensembles for a physical system is its \textbf{ensemble space}. Formally, an ensemble space is a $\mathsf{T}_0$ second countable topological space where each element is called an ensemble.
	\end{axiom}
	
	\begin{justification}
		In experimental settings, preparation procedures never prepare a system exactly in the same configuration. Experimental results, then, are always in terms of statistical preparations and statistical measurements. A physical theory must be able to talk about the possible statistical descriptions within the theory. States, then, can be understood as ensembles, idealized statistical descriptions, as those are what is connected to experimental practice.
		
		Equivalently, reproducibility is a basic requirement of a physical theory. A physical law, then, must be understood as describing a relationship that always exists whenever the same set of circumstances is replicated. Given that we need to always be able to replicate those circumstances ``one more time'', the relationship is about countably infinite preparations and results: ensembles. Therefore, to the extent that physics is about reproducible experimental results, the basic theoretical description of a system is in terms of ensembles. This justifies the use of ensembles as the fundamental object to describe the state of a system.\footnote{Note that reproducibility also already implies that all properties that characterize an ensemble must be relative to the procedure. If the properties depended, for example, on absolute space or absolute time, then different practitioners would not be able to prepare the same ensemble.}
		
		Ensembles are experimentally defined objects, and therefore they are possibilities of an experimental domain. This means that an ensemble space is a $\mathsf{T}_0$ second countable topological space where each element is an ensemble and the topology is induced by the verifiable statements.
	\end{justification}
\end{mathSection}

We should now verify that our three standard cases satisfy the axiom of ensemble.

\begin{mathSection}
\begin{prop}
	Classical discrete, classical continuous and quantum ensemble spaces satisfy the axiom of ensemble.
\end{prop}

\begin{proof}
	For the classical discrete case, including all infinite convex combinations, we have the subset of $\ell^1$ of all non-negative sequences that sum to one. Similarly, for the classical continuous case we have the subset of $L^1$ that corresponds to non-negative distributions that integrate to one. Both of these spaces, with the subspace topology, are separable, admit a countable orthonormal basis and can be given a topology that is $\textsf{T}_0$ and second countable. Note that, in case the correct formulation is in terms of square integrable functions, instead of simply integrable functions, the space is still $\textsf{T}_0$ and second countable.
	
	For the quantum case, the Hilbert space with its standard topology is also separable, will admit a countable orthonormal basis and can be given a topology that is $\textsf{T}_0$ and second countable. A density operator will be fully defined by its result on the basis, which means the space of ensembles is also a vector space with a countable basis and can be given a topology that is $\textsf{T}_0$ and second countable.
\end{proof}
\end{mathSection}

\section{Axiom of mixture and convex structure}

In this section we are going to see how the ability to perform statistical mixtures leads to a convex structure for ensemble spaces. Only mixtures of finitely many elements are guaranteed to exist, and the topology will tell us which infinite mixtures are possible. The convex structure also gives us a basic notion to compare ensembles: one ensemble can be a component of another if the second can be seen as a mixture of the first with something else. Two ensembles are separate if they have no common component.

As we saw before, an ensemble is the collection of all outputs of a preparation procedure. The idea is that we can always combine preparation procedures selecting the output among them with a given probability distribution. This statistical mixture is another preparation procedure which will correspond to an ensemble. The ensemble space of any physical theory, then, must allow statistical mixtures, which leads to a convex structure.

\begin{mathSection}
\begin{defn}
	Given a real number $p \in [0,1]$, its complement is defined as $\bar{p} = 1-p$.
\end{defn}

\begin{axiom}[Axiom of mixture]\label{pm-es-axiomMixture}
	The statistical mixture of two ensembles is an ensemble. Formally, an ensemble space $\Ens$ is equipped with an operation $+ : [0, 1] \times \Ens \times \Ens \to \Ens$ called \textbf{mixing}, noted with the infix notation $p \ens[a] + \bar{p} \ens[b]$, with the following properties:
	\begin{itemize}
		\item \textbf{Continuity}: the map $+(p, \ens[a], \ens[b])  \to p \ens[a] + \bar{p} \ens[b]$ is continuous (with respect to the product topology of $[0, 1] \times \Ens \times \Ens$)
		\item \textbf{Identity}: $1 \ens[a] + 0 \ens[b] = \ens[a]$
		\item \textbf{Idempotence}:  $p \ens[a] + \bar{p} \ens[a] = \ens[a]$ for all $p \in [0,1]$
		\item \textbf{Commutativity}: $p \ens[a] + \bar{p} \ens[b] = \bar{p} \ens[b] + p \ens[a]$ for all $p \in [0,1]$
		\item \textbf{Associativity}: $p_1 \ens_1 + \bar{p}_1\left(\frac{p_2}{\bar{p}_1}\ens_2 + \frac{p_3}{\bar{p}_1}\ens_3\right) =  \bar{p}_3\left(\frac{p_1}{\bar{p}_3} \ens_1 +  \frac{p_2}{\bar{p}_3}\ens_2\right) + p_3 \ens_3$ where $p_1, p_3 \in [0,1)$ and $p_1 + p_3 \leq 1$ and $p_2 = 1 - p_1 - p_3$
	\end{itemize}
\end{axiom}

\begin{justification}
	This axiom captures the ability to create a mixture merely by selecting between the output of different processes. Let $\ens_1$ and $\ens_2$ be two ensembles that represent the output of two different processes $P_1$ and $P_2$. Let a selector $S_p$ be a process that outputs two symbols, the first with probability $p$ and the second with probability $\bar{p}$. Then we can create another process $P$ that, depending on the selector, outputs either the output of $P_1$ or $P_2$. All possible preparations of such a procedure will form an ensemble. Therefore we are justified in equipping an ensemble space with a mixing operation that takes a real number from zero to one, and two ensembles.
	
	Given that mixing represents an experimental relationship, and all experimental relationships must be continuous in the natural topology, mixing must be a continuous function. In general, the mixing coefficient $p$ corresponds to a value from a continuously ordered quantity between zero and one, as defined in the previous chapter, and therefore the natural topology is the one of the reals.\footnote{It may be argued that rational numbers could be prepared exactly, as one may design a procedure that, for example, alternates the selection deterministically. Therefore one could have a countable subset of topologically isolated ensembles. It is not clear whether this would create problems or not. Since the topology of the reals would still be required as a subspace topology anyway, we leave investigating this case to future work.} This justifies continuity.

	If $p=1$, the output of $P$ will always be the output of $P_1$. This justifies the identity property. If $P_1$ and $P_2$ are the same process, then the output of $P$ will always be the output of $P_1$. This justifies the idempotence property. The order in which the processes are given does not matter as long as the same probability is matched to the same process. The process $P$ is identical under permutation of $P_1$ and $P_2$. This justifies commutativity. If we are mixing three processes $P_1$, $P_2$ and $P_3$, as long as the final probabilities are the same, it does not matter if we mix $P_1$ and $P_2$ first or $P_2$ and $P_3$. This justifies associativity.
\end{justification}

\begin{coro}
	An ensemble space is a convex space.
\end{coro}

\begin{proof}
	The properties of the axiom of mixture match the basic definition of convex spaces. For example, see  \href{https://ncatlab.org/nlab/show/convex+space}{https://ncatlab.org/nlab/show/convex+space} or \href{https://arxiv.org/abs/0903.5522}{arXiv:0903.5522}.  The notation and terminology will be slightly different to better map to physics ideas. 
\end{proof}

\end{mathSection}

As we progress through the details of the theory, we will see that all linear structures in physics are, in one way or another, manifestations of this basic structure. For example, the linearity of the Hilbert space in quantum mechanics is connected to the linearity of density operators and expectations. 

Before proceeding, we should now check that the axiom of mixture is satisfied by the standard cases.

\begin{mathSection}
\begin{prop}
	Discrete classical ensemble spaces, continuous classical ensemble spaces and quantum ensemble spaces satisfy the axiom of mixture.
\end{prop}

\begin{proof}
	The space $\Ens$ of probability measures, discrete or continuous, is a convex subset of the topological vector space of signed finite measures. It is therefore closed under convex combinations: if $\ens[a], \ens[b] \in \Ens$ are probability measures, then $p \ens[a] + \bar{p} \ens[b]$ is a probability measure. The properties of mixing are inherited from the properties of linear combinations, which include continuity. Therefore the discrete and continuous classical ensemble spaces satisfy the axiom of mixture.
	
	Similarly, the space of positive semi-definite self-adjoint operators with trace one is a convex subset of the topological vector space of self-adjoint operators. Therefore it is closed under convex combinations, which are continuous in the given topology, and it will satisfy the axiom of mixture.
\end{proof}
\end{mathSection}

\subsection{Finite and infinite mixtures}

The axiom of mixture only guarantees the existence of a mixture between two ensembles. We can mix elements recursively and extend the operation to finitely many ensembles. Commutativity and associativity make these mixtures independent of the mixing order, such that they depend only on the ensembles chosen and the mixing coefficients associated to each ensemble.

\begin{mathSection}
\begin{defn}[Finite mixture]
	Let $\{e_i\}_{i=1}^{n} \subseteq \Ens$ be a finite subset of ensembles and $\{p_i\} \in [0,1]$ be a finite set of coefficients such that $\sum_{i=1}^{n} p_i = 1$, then the \textbf{finite mixture}, noted $\ens[a] = \sum_{i=1}^{n} p_i \ens_i$, is defined to be
	\begin{equation}
		\begin{split}
	p_1 \ens_1 + (1-p_1)\left( \tfrac{p_2}{1-p_1} \ens_2 + \tfrac{1-p_1-p_2}{1-p_1} \left( \tfrac{p_3}{1-p_1-p_2} \ens_3 +  \tfrac{1-\sum_{i=1}^{3}p_i}{1-p_1-p_2}\Bigl(\ldots \right.\right. \\
\left.\left.\left.+\tfrac{1-\sum_{i=1}^{n-2}p_i}{1-\sum_{i=1}^{n-3}p_i}\left(\tfrac{p_{n-1}}{1- \sum_{i=1}^{n-2}p_i} \ens_{n-1} + \tfrac{p_n}{1- \sum_{i=1}^{n-2}p_i}\ens_n\right)\right) \right) \right).
		\end{split}
	\end{equation}
\end{defn}

\begin{check}
	For the definition to work, we need to make sure that the final ensemble does not depend on the order of mixing. We first check with three elements. Let $p_1, p_2, p_3 \in [0,1]$ with $p_1+p_2+p_3=1$. By commutativity, we can switch the second with the third element:
	\begin{equation}
	\begin{aligned}
		p_1 \ens[e]_1 + p_2 \ens[e]_2 + p_3 \ens_3 &= p_1 \ens_1 + \bar{p}_1\left(\frac{p_2}{\bar{p}_1}\ens_2 + \frac{p_3}{\bar{p}_1}\ens_3\right) = p_1 \ens_1 + \bar{p}_1\left(\frac{p_3}{\bar{p}_1} \ens_3 +  \frac{p_2}{\bar{p}_1}\ens_2\right) \\
		&= p_1 \ens[e]_1 + p_3 \ens[e]_3 + p_2 \ens_2.
	\end{aligned}
	\end{equation}
	
	Then, by commutativity and associativity, we can switch the first with the third element:
	\begin{equation}
		\begin{aligned}
	 p_1 \ens[e]_1 + p_2 \ens[e]_2 + p_3 \ens_3 &= p_1 \ens_1 + \bar{p}_1\left(\frac{p_2}{\bar{p}_1}\ens_2 + \frac{p_3}{\bar{p}_1}\ens_3\right) = p_1 \ens_1 + \bar{p}_1\left(\frac{1-p_1-p_3}{\bar{p}_1}\ens_2 + \frac{p_3}{\bar{p}_1}\ens_3\right) \\
	 &= \bar{p}_3\left(\frac{p_1}{\bar{p}_3} \ens_1 + \frac{1-p_1-p_3}{\bar{p}_3}\ens_2\right) + p_3 \ens_3 = p_3 \ens_3 + \bar{p}_3\left(\frac{p_1}{\bar{p}_3} \ens_1 + \frac{p_2}{\bar{p}_3}\ens_2\right) \\
	 &= p_3 \ens[e]_3 + p_2 \ens[e]_2 + p_1 \ens_1.
		\end{aligned}
	\end{equation}
	Since switching the first with the second is equivalent to switching the second with the third and the third with the first, we can reach all permutations.
	
	Note that the definition is recursive, therefore we can use proof by induction. The base case is a sequence of two elements. By commutativity, the order does not matter. Given a sequence of $n$ elements, the inductive hypothesis is that the order does not matter for the last $n-1$ elements. Therefore, it suffices to show that we can switch the first and the second element. Note that we can sum the last $n-2$ elements, thus converting this to a problem of three elements. We proved before that we can switch the first with the second element, and we can now re-expand the third element into the full sequence. Therefore, the order of mixing does not matter for any finite mixture. 
\end{check}

\begin{remark}
	Note that we can collect and expand convex combinations into other convex combinations. Because coefficients always sum to one, when breaking the expression in two, we can always calculate the new coefficients from one part. For example, $p_a \ens[a] + p_b \ens[b] + p_c \ens[c] + p_d \ens[d] = p(\frac{p_a}{p} \ens[a] + \frac{p_b}{p} \ens[b]) + \bar{p}(\frac{p_c}{\bar{p}} \ens[c] + \frac{p_d}{\bar{p}} \ens[d])$ where $p = p_a + p_b$ is defined only on the left part. Since $1 = p + \bar{p} = p_a +p_b + p_c + p_d$, we automatically have that $\bar{p} = p_c + p_d$.
\end{remark}
\end{mathSection}

While mixtures of finitely many elements are always guaranteed to exist, the extension to mixture of infinite elements is not. First of all, an infinite mixture of ensembles with finite entropy does not necessarily have finite entropy.\footnote{For details, see \href{https://arxiv.org/pdf/1212.5630.pdf}{J. Stat. Mech. (2013) P04010}.} Secondly, it may lead to infinite expectation values which make ensembles not physically meaningful. For example, suppose we put $i$ particles in a box according to the distribution $\frac{6}{\pi^2 i^2}$ given that $\sum_{i=1}^{\infty} \frac{6}{\pi^2 i^2}=1$. The expectation $\sum_{i=1}^{\infty} \frac{6}{\pi^2 i^2} i$ diverges. Given that every finite preparation will have a finite expectation value, the actual implementation of that ensemble will necessarily give us a stream of preparations whose number of particles must keep increasing. No finite statistics is representative of the ensemble and, worst of all, the finite statistics will necessarily have averages of arbitrarily large differences. We therefore cannot simply look at the coefficients to understand whether the infinite mixture gives us a valid ensemble or not.

Whether an infinite mixture $\sum_{i=1}^\infty p_i \ens_i$ converges or not in the ensemble space is therefore determined by the topology (i.e. experimental verifiability). The axiom guarantees only finite mixtures and we let the closure of the topology handle the limits.

\begin{mathSection}
\begin{defn}[Infinite mixture]
	Let $\{e_i\}_{i=1}^{\infty} \subseteq \Ens$ be a sequence of ensembles and $\{p_i\} \in [0,1]$ be a sequence of coefficients such that $\sum_{i=1}^{\infty} p_i = 1$. Then the ensemble $\ens[a]$ is an \textbf{infinite mixture} of those ensembles if it is a topological limit of the sequence of finite mixtures $\sum_{i=1}^{n} \frac{p_i}{p_n} \ens_i$, where $p_n = \sum_{i=1}^{n} p_i$. If the infinite mixture is unique, we write $\ens[a] = \sum_{i=1}^{\infty} p_i \ens_i$.
\end{defn}

\begin{remark}
	We will see later that the entropy constrains the topology to be Hausdorff, therefore all infinite mixtures, if they exist, are unique.
\end{remark}
\end{mathSection}

It is unclear whether commutativity and associativity extend, or should extend, to infinite mixtures. For series, \href{https://en.wikipedia.org/wiki/Unconditional_convergence}{unconditional convergence} defines convergence that does not depend on infinite reordering, but it is unclear even whether this is a desirable property. Additionally, we would expect that if an infinite mixture is possible, any submixture should converge as well. That is, if $\sum_{i=1}^{\infty} p_i \ens_i$ converges, then $\sum_{i \in I} \frac{p_i}{p_I} \ens_i$ converges for all $I \subseteq \mathbb{N}$ where $p_I = \sum_{i \in I} p_i$.


We do know that the convex structure is not enough to guarantee the above property, as shown by the following counterexample provided \href{https://math.stackexchange.com/questions/5085283/does-a-convex-subcombination-of-a-convergent-convex-combination-converge}{on stack exchange}. Consider $\mathbb{R}$. It is a metrizable second-countable topological vector space, which means it is a convex space with a topology such that the mixing operation is continuous. It satisfies the axiom of ensemble and the axiom of mixture. Let $p_i = \frac{6}{\pi^2 i^2}$, which means $\sum_i p_i = 1$ and $\ens_i = (-1)^i i$. We have
\begin{equation*}
	\sum_{i=1}^{\infty} p_i \ens_i = \frac{6}{\pi^2} \sum_{i=1}^{\infty} \frac{(-1)^i}{i} = - \frac{6}{\pi^2} \ln 2
\end{equation*}
which means it converges. However, let $I = \{2,4,6,\dots\}$. We have
\begin{equation*}
	\begin{aligned}
		p_I &= \sum_{k=1}^{\infty} p_{2k} = \sum_{k=1}^{\infty} \frac{6}{\pi^2 4 k^2} = \frac{1}{4} \\
		\sum_{i \in I} \frac{p_i}{p_I} \ens_i &= 4 \sum_{i \in I} p_i \ens_i = 4 \sum_{k=1}^{\infty} \frac{6}{\pi^2 (4k^2)} 2k = \frac{12}{\pi^2} \sum_{k=1}^{\infty} \frac{1}{k} \to \infty.
	\end{aligned}
\end{equation*}
Therefore the convex combination of all even elements of the series diverges.

Note that the above counterexample works because the series is effectively the sum of two divergent series. The axiom of entropy will force the ensemble space to have a bounded intersection with every affine line. Therefore the above problem would not exist, because one cannot produce a divergent series from a bounded interval of the real line. It is not clear, though, whether this is enough to show that submixtures of convergent infinite mixtures converge. We leave this as conjecture \ref{pm-es-conjSubmixturesConverge}.

TODO: add conjecture for definition of convex integrals

\subsection{Common components and separateness}

The convex structure allows us to characterize ensembles based on whether they can be mixed into one another. For example, we can ask whether an ensemble is or is not the mixture of some other ensembles; or whether two ensembles can be expressed as a mixture of a common component.

\begin{mathSection}
\begin{defn}
	Let $\Ens$ be an ensemble space. Let $\ens[a] = \sum_i p_i \ens_i$ where $\ens[a], \ens_i \in \Ens$ and $p_i \in (0,1]$ such that $\sum_i p_i = 1$. We say that $\ens[a]$ is a \textbf{mixture} of $\{\ens_i\}$, each $\ens_i$ is a \textbf{component} of $\ens[a]$ and each $p_i$ is a \textbf{mixture coefficient}.
\end{defn}

\begin{figure}[H]
	\centering
	\def\angle{10}
	\def\radius{1.2}
	\begin{tikzpicture}		
		%\draw [help lines] (0,0) grid (10,3);
		\coordinate (a)      at (0,0);
		\coordinate (aab)    at (0.5,0.75);
		\coordinate (ab)    at (1,1.5);
		\coordinate (b)      at (2,3);
		
		\draw[-, thick] (a) -- (b);
		\filldraw [black] (a) circle (1pt) node[left] {$\ens[a]$};
		\filldraw [black] (ab) circle (1pt) node[left] {$\frac{1}{2}\ens[a] + \frac{1}{2}\ens[b]$};
		\filldraw [black] (aab) circle (1pt) node[right] {$\frac{3}{4}\ens[a] + \frac{1}{4}\ens[b]$};
		\filldraw [black] (b) circle (1pt) node[right] {$\ens[b]$};
		
		\coordinate (center) at (4,1.5);
		
		\draw[fill=ensembleFill, thick] ($(center)+\radius*({cos(0+\angle)},{sin(0+\angle)})$) -- ($(center)+\radius*({cos(120+\angle)},{sin(120+\angle)})$) -- ($(center)+\radius*({cos(240+\angle)},{sin(240+\angle)})$) -- cycle;
		
		\coordinate (c)    at (6.5,0.5);
		\coordinate (d)    at (7.5,1.5);
		\coordinate (e)    at (8.5,2.5);
		
		\draw[-, thick] (c) -- (d);
		\draw[dashed, thick] (d) -- (e);
		\filldraw [black] (c) circle (1pt) node[left] {$\ens[c]$};
		\filldraw [black] (d) circle (1pt) node[right] {$\ens[d]$};
		\filldraw [black] (e) circle (1pt) node[right] {$\ens[e]$};
	\end{tikzpicture}
	%\includegraphics[width=0.5\textwidth]{tempimages/ConvexExamples.jpg}
\end{figure}

\begin{remark}
	In terms of the convex space, all the mixtures of two ensembles correspond to the segment between them; all the mixtures of three ensembles correspond to the triangle formed by the three elements and so on. An ensemble $\ens[c]$ is a component of a different ensemble $\ens[d]$ if the segment connecting $\ens[c]$ and $\ens[d]$ can be extended past $\ens[d]$. If two elements are not a component of each other, then they are the extreme points of the line that goes through the two. That is, the segment cannot be extended.
\end{remark}

\begin{figure}[H]
	\centering
	\begin{tikzpicture}		
		\coordinate (a)   at (0,0);
		\coordinate (2ab)   at (1,0);
		\coordinate (a2b)   at (2,0);
		\coordinate (b)   at (3,0);
		
		\draw[-, thick] (a) -- (b);
		\filldraw [black] (a) circle (1pt) node[above left] {$\ens[a]$};
		\filldraw [black] (2ab) circle (1pt) node[above] {$\frac{2}{3}\ens[a] + \frac{1}{3}\ens[b]$};
		\filldraw [black] (a2b) circle (1pt) node[below] {$\frac{1}{3}\ens[a] + \frac{2}{3}\ens[b]$};
		\filldraw [black] (b) circle (1pt) node[above right] {$\ens[b]$};
	\end{tikzpicture}
%	\includegraphics[width=0.4\textwidth]{tempimages/ComponentNotOrder.jpg}
\end{figure}

\begin{remark}
	Note that two ensembles can be components of each other. Consider $\frac{2}{3} \ens[a] + \frac{1}{3} \ens[b]$ and $\frac{1}{3} \ens[a] + \frac{2}{3} \ens[b]$. We can write $\frac{2}{3} \ens[a] + \frac{1}{3} \ens[b] = \frac{1}{2}\left(\frac{1}{3} \ens[a] + \frac{2}{3} \ens[b]\right) + \frac{1}{2} \ens[a]$ and $\frac{1}{3} \ens[a] + \frac{2}{3} \ens[b] = \frac{1}{2}\left(\frac{2}{3} \ens[a] + \frac{1}{3} \ens[b]\right) + \frac{1}{2} \ens[b]$ (they are both midpoints along each other). Therefore a component is not necessarily ``smaller'' or ``better defined'' than the mixture. Mathematically, ``being a component of'' is not a partial order. It is reflexive and transitive, but it is not antisymmetric. In practical terms, we need something else to tell us whether we are, for example, taking a limit with components that become ``smaller and smaller.''
\end{remark}
\end{mathSection}

We can also characterize some ensembles based on what other ensembles they can admit as components. An extreme point is an ensemble that has only itself as a component. For example, a pure state will be an extreme point as it cannot be expressed as a mixture of any other states. Conversely, an internal point is an ensemble that admits any other ensemble as a component. For example, in a finite discrete classical space, the uniform distribution over all cases can be seen as the mixture of any other distribution with something else. A boundary point is an ensemble that is not  an internal point. The figure helps visualize the properties.

\begin{mathSection}
\begin{defn}
	Let $\Ens$ be an ensemble space. An \textbf{extreme point} $\ens \in \Ens$ is an ensemble that has no component distinct from itself. That is, there is no $\ens[a] \in \Ens\setminus\{ \ens \}$ such that $\ens = p \ens[a] + \bar{p} \ens[b]$ for some $p \in (0,1]$ and $\ens[b] \in \Ens$. An \textbf{internal point} $\ens \in \Ens$ is an ensemble for which every ensemble is a component. That is, for every $\ens[a] \in \Ens$ there is always $\ens[b] \in \Ens$ and $p \in (0,1]$ such that $\ens = p \ens[a] + \bar{p} \ens[b]$. A \textbf{boundary point} is any ensemble that is not an internal point.
\end{defn}

\begin{remark}
	The notion of internal point parallels \href{https://en.wikipedia.org/wiki/Algebraic_interior}{the similar notion} in topological vector spaces.
\end{remark}

\begin{figure}[H]
	\centering
	\begin{tikzpicture}		
		%\draw [help lines] (0,0) grid (10,3);
		
		\coordinate (center) at (1.5,1.2);
		\def\angle{90}
		\def\radius{1.5}
		\draw[fill=ensembleFill, thick] ($(center)+\radius*({cos(0+\angle)},{sin(0+\angle)})$) node[right] {$\ens[a]$} -- ($(center)+\radius*({cos(120+\angle)},{sin(120+\angle)})$) node[midway, left] {$\ens[b]$} -- ($(center)+\radius*({cos(240+\angle)},{sin(240+\angle)})$) -- cycle  ;
		\filldraw [black] ($(center)+\radius/2*({cos(60+\angle)},{sin(60+\angle)})$) circle (1pt);
		\filldraw [black] ($(center)+\radius*({cos(0+\angle)},{sin(0+\angle)})$) circle (1pt);
		\filldraw [black] ($(center)-\radius*(0,1/4)$) circle (1pt) node[above] {$\ens[c]$};
		
		\coordinate (center) at (5,1.5);
		\def\radius{1.25}
		\draw [fill=ensembleFill, thick] (center) circle (\radius);
		\filldraw [black] ($(center)+\radius*({cos(45)},{sin(45)})$) circle (1pt) node[above right] {$\ens[a]$};
		\filldraw [black] ($(center)-\radius*(0,1/4)$) circle (1pt) node[above] {$\ens[c]$};
	\end{tikzpicture}
	%\includegraphics[width=0.8\textwidth]{tempimages/InteriorExteriorPoints.jpg}
	\caption{In both spaces, $\ens[a]$ is an extreme point while $\ens[c]$ is an internal point; $\ens[b]$ is a boundary point, but is not an extreme point (it is the mixture of the vertices on the same side). On the circle, all boundary points are extreme points.}
\end{figure}

\begin{prop}
	The set of all internal points $I_{\Ens}$ is a convex set.
\end{prop}
\begin{proof}
Let $\ens_1, \ens_2 \in I_{\Ens}$ be two internal points of $\Ens$. Since $\ens_1$ and $\ens_2$ are internal points, given any $\ens[a] \in \Ens$ we can find $\ens[b]_1, \ens[b]_2 \in \Ens$ such that 
\begin{equation}
	\begin{aligned}
	\ens_1 & =p_1 \ens[a]+\bar{p}_1 \ens[b]_1 \\
	\ens_2 & =p_2 \ens[a]+\bar{p}_2 \ens[b]_2
	\end{aligned}
\end{equation}
for some $p_1, p_2 \in (0,1]$. Now let $p \in [0,1]$ and $\ens = p \ens_1+\bar{p} \ens_2$. We have:
\begin{equation}
\begin{aligned}
	\ens = p \ens_1+\bar{p} \ens_2 &= p(p_1 \ens[a]+\bar{p}_1 \ens[b]_1) + \bar{p} (p_2 \ens[a]+\bar{p}_2 \ens[b]_2)\\
	&=\left(p p_1+\bar{p} p_2\right) \ens[a]+\left(p \bar{p}_{1} \ens[b]_1+\bar{p} \bar{p}_{2} \ens[b]_2\right) \\
	&=\lambda \ens[a] + \bar{\lambda} \left(\frac{p \bar{p}_{1}}{\bar{\lambda}} \ens[b]_1+\frac{\bar{p} \bar{p}_{2}}{\bar{\lambda}} \ens[b]_2\right) = \lambda \ens[a] + \bar{\lambda} \ens[b] \\
\end{aligned}
\end{equation}
where $\lambda = p p_1+\bar{p} p_2$ and $\ens[b] = \frac{p \bar{p}_{1}}{\bar{\lambda}} \ens[b]_1+\frac{\bar{p} \bar{p}_{2}}{\bar{\lambda}} \ens[b]_2$. Therefore, given any $\ens[a] \in \Ens$, we can find a $\ens[b] \in \Ens$ such that $\ens = \lambda \ens[a] + \bar{\lambda} \ens[b]$ for some $\lambda \in (0,1]$. This means the convex combination of internal points is an internal point, and $I_{\varepsilon}$ is a convex set.
\end{proof}
\end{mathSection}

It is an open question whether this result generalizes to infinite mixtures. That is, whether the infinite mixture of internal points is still an internal point. To generalize the previous proof to the infinite case, one would need to show that the infinite mixture of $\ens[b]_i$ converges. If conjecture \ref{pm-es-conjSubmixturesConverge} is true, that is if submixtures of convergent infinite mixtures converge, then the infinite mixture of $\ens[b]_i$ would converge as it is a submixture of $\ens$. We leave this as conjecture \ref{pm-es-conjAlgebraicInteriorSigmaConvex}.

For topological vector spaces, the algebraic interior and the topological interior are related. For example, if $A$ is a convex subset of a TVS with non-empty topological interior, then the algebraic and topological interior coincide. It is an open question how many of these results hold for topological convex spaces as well. As an example, we would like to prove the following.

TODO: reorganize

\begin{conj}
	Let $\Ens$ be an ensemble space. The set of boundary points $B_{\Ens}$ is not necessarily a closed set and therefore $I_{\Ens}$ is not necessarily an open set.
\end{conj}
\begin{proof}
	Let $\Ens = \{ p_i \in \ell^1 \, | \,  \sum p_ i =1, \; p_i \in [0,1]\}$ be the set of probability distributions over countably infinitely many elements with the topology of $\ell^1$. Let $\ens \in I_{\Ens}$ be an interior point of $\Ens$. Since $\ens$ is in the interior, it corresponds to a probability distribution $p_i$ such that $p_i \in (0,1)$ for all $i$.
	
	Let $B_{r}(\ens) \subseteq \ell^1$ be an open ball of radius $r$ centered around $\ens$. Since $\sum p_i = 1$, there must be a $j$ such that $p_j < \frac{r}{2}$. Suppose, without loss of generality, that $j \neq 1$. Now consider the probability distribution given by $\lambda_i = (p_1 + p_j, p_2, \dots, p_{j-1}, 0, p_{j+1}, \dots)$. Since $\lambda_j = 0$, then $\lambda_i$ is a boundary point of $\Ens$. The distance between the two distribution will be $\sum|p_i - \lambda_i| = |p_j| + |-p_j| = 2 p_j < r$, which means $\lambda_i \in B_{r}(\ens)$. Therefore, the open ball around any internal point will contain a boundary point. Therefore the interior of $I_{\Ens}$ is the empty set, and $I_{\Ens}$ is not an open set.
\end{proof}

One may be able to show that the limit of a sequence of boundary points cannot be an interior point in the topological sense.

We now define the notions of separateness: two ensembles are separate, noted $\ens[a] \separate \ens[b]$ if they do not have a common component. That is, they are not a mixture of a common ensemble. In classical ensemble spaces, this is equivalent to probability measures with disjoint support, which is the extension of the concept. Separateness is a useful concept to characterize the relationship between ensembles as it has some useful properties. Most of all, it is an irreflexive symmetric relation: nothing is separate from itself and if $\ens[a] \separate \ens[b]$, then $\ens[b] \separate \ens[a]$.\footnote{Note that orthogonality in inner product vector spaces is also an irreflexive symmetric relationship.}

\begin{mathSection}
\begin{defn}
	Let $\Ens$ be an ensemble space and $\ens[a], \ens[b] \in \Ens$. We say that they \textbf{have a common component} if we can find $\ens[c] \in \Ens$, the common component, such that $\ens[a] = p_1 \ens[c] + \bar{p}_1 \ens_1$ and $\ens[b] = p_2 \ens[c] + \bar{p}_2 \ens_2$ for some $\ens_1, \ens_2 \in \Ens$ and $p_1, p_2 \in (0,1]$. Otherwise, we say they \textbf{have no common component}, or are \textbf{separate}, noted $\ens[a] \separate \ens[b]$. Two ensembles have a common component in $A \subseteq \Ens$ if the common component can be found in $A$, and are separate in $A$ if there is none. Two sets of ensembles $A, B \subseteq \Ens$ are separate if all the elements of one are separate from all the elements of the other. That is, $A \separate B$ if $\ens[a] \separate \ens[b]$ for all $\ens[a] \in A$ and $\ens[b] \in B$.
\end{defn}

\begin{figure}[H]
	\centering
	\begin{tikzpicture}		
		%\draw [help lines] (0,0) grid (10,3);
		
		\coordinate (a) at (1.0,1.0);
		\coordinate (c) at (3.0,0.7);
		\coordinate (b) at (2.4,1.9);
		\def\aover{0.3}
		\def\bover{0.6}
		\filldraw [black] (a) circle (1pt) node[above right] {$\ens[a]$};
		\filldraw [black] (b) circle (1pt) node[above right] {$\ens[b]$};
		\filldraw [black] (c) circle (1pt) node[above right] {$\ens[c]$};
		\filldraw [black] ($(a)!-\aover!(c)$) circle (1pt) node[above right] {$\ens_1$};
		\filldraw [black] ($(b)!-\bover!(c)$) circle (1pt) node[above right] {$\ens_2$};
		\draw[thick] ($(a)!-\aover!(c)$) -- (c) -- ($(b)!-\bover!(c)$)  ;
	\end{tikzpicture}
	%\includegraphics[width=0.3\textwidth]{tempimages/CommonComponent.jpg}
\end{figure}

\begin{remark}
	If two ensembles $\ens[a]$ and $\ens[b]$ have a common component $\ens[c]$, then the ensemble space contains a triangle where $\ens[c]$ is a vertex and $\ens[a]$ and $\ens[b]$ are points on the sides that connect to $\ens[c]$.
\end{remark}

\begin{coro}
	The previous definitions obey the following:
	\begin{enumerate}
		\item every ensemble is a component of itself
		\item if $\ens[a]$ is a component of $\ens[b]$, then $\ens[a]$ and $\ens[b]$ have a common component and therefore they are not separate
		\item separateness is an irreflexive symmetric relation
		\item an ensemble is an extreme point if and only if it is separate from all other ensembles
		\item an ensemble is an internal point if and only if it is not separate from any ensemble
		\item if two ensembles are separate, then they are boundary points
	\end{enumerate}
\end{coro}

\begin{proof}
	1. Since by idempotence $\ens = p \ens + \bar{p} \ens$ for any $p$, then every ensemble is a mixture of itself, and therefore it is a component of itself.
	
	2. By idempotence, we can write $\ens[a] = p_1 \ens[a] + \bar{p}_1 \ens[a]$ for some $p_1 \in (0,1]$. Since $\ens[a]$ is a component of $\ens[b]$, we can write $\ens[b] = p_2 \ens[a] + \bar{p}_2 \ens[e]_2$ for some $p_2 \in (0, 1]$ and $\ens[e]_2 \in \Ens$. Therefore $\ens[a]$ and $\ens[b]$ have $\ens[a]$ as a common component.
	
	3. Since every ensemble is a component of itself, every ensemble has a common component with itself and therefore is not separate from itself. This proves that separateness is irreflexive. The definition of common component is symmetric and therefore so is separateness.
	
	4. An extreme point has only itself as a component, therefore it can have a common component only with itself.
	
	5. Every ensemble is a component of an internal point, therefore every ensemble is not separate from an internal point
	
	6. Since an internal point cannot be separate from any ensemble, then two ensembles that are separate are not internal points, and therefore are boundary points.
\end{proof}

\begin{prop}
	Let $\Ens$ be a discrete or continuous classical ensemble space and let $\ens[a], \ens[b] \in \Ens$ be two probability measures over the corresponding sample space $X$. Then $\ens[a] \separate \ens[b]$ if and only if they have disjoint support.
\end{prop}

\begin{proof}
	Note that the support of a convex combination of probability measures is the union of the support of the measures. Since we are restricting ourselves to probability measures that are absolutely continuous with respect to a measure $\mu$, if a probability measure $\ens[a]$ has support $U$, any probability measure $\ens[b]$ with support on a compact subset $V \subseteq U$ such that $\mu(V) \neq 0$ is a component of $\ens[a]$.
	
	Let $\ens[a], \ens[b] \in \Ens$ be two probability measures over the sample space $X$ that are absolutely continuous with respect to the corresponding $\mu$. Suppose $\ens[a]$ and $\ens[b]$ have overlapping support. The we can find a compact subset $U$ such that $\mu(U) \neq 0$ and is a subset of the intersection of their supports. Therefore, we can find a probability measure that is a component of both. Conversely, suppose they have disjoint support. Then they cannot have a common component, as the support of the common component would have to be a non-empty subset of both supports.
\end{proof}
\end{mathSection}

A key property of separateness is its relationship with mixtures. If an ensemble is separate from a mixture of two elements, it is separate from both elements and all their mixtures.

\begin{mathSection}
\begin{prop}[Separateness extends to all mixtures]\label{pm-es-separateExtendsMixtures}
	Let $\ens,\ens_1,\ens_2 \in \Ens$. If $\ens$ has no common component with a mixture of $\ens_1$ and $\ens_2$ then it has no common component with any mixture of $\ens_1$ and $\ens_2$ and with either $\ens_1$ or $\ens_2$. That is, if $\ens \separate p \ens_1 + \bar{p} \ens_2$ for some $p \in (0, 1)$ then $\ens \separate p \ens_1 + \bar{p} \ens_2$ for all $p \in [0, 1]$.
\end{prop}

\begin{figure}[H]
	\centering
	\begin{tikzpicture}		
		%\draw [help lines] (0,0) grid (4,4);
		
		\coordinate (e) at (1.2,0.8);
		\coordinate (e1) at (0.2,2.5);
		\coordinate (e2) at (3.0,2.5);
		\coordinate (a) at ($(e1)!0.4!(e2)$);
		\coordinate (b) at ($(a)!0.5!(e2)$);
		\coordinate (c) at (3.5,0.3);
		\coordinate (d) at ($(b)!-0.4!(c)$);
		\coordinate (f) at ($(e)!-0.25!(c)$);
		\coordinate (g) at ($(a)!-0.195!(c)$);
		\filldraw [black] (e) circle (1pt) node[below] {$\ens$};
		\filldraw [black] (e1) circle (1pt) node[left] {$\ens_1$};
		\filldraw [black] (e2) circle (1pt) node[right] {$\ens_2$};
		\filldraw [black] (a) circle (1pt) node[above right] {$\ens[a]$};
		\filldraw [black] (b) circle (1pt) node[above right] {$\ens[b]$};
		\filldraw [black] (c) circle (1pt) node[above right] {$\ens[c]$};
		\filldraw [black] (d) circle (1pt) node[above right] {$\ens[d]$};
		\filldraw [black] (f) circle (1pt) node[left] {$\ens[f]$};
		\filldraw [black] (g) circle (1pt) node[above left] {$\ens[g]$};
		\draw[thick] (d) -- (e1) -- (e2);
		\draw[thick] (d) -- (c) -- (f);
		\draw[thick] (g) -- (c);
	\end{tikzpicture}
	%\includegraphics[width=0.3\textwidth]{tempimages/DistinctAndMixture.jpg}
\end{figure}

\begin{proof}
	Let $\ens \separate \ens[a] = p \ens_1 + \bar{p} \ens_2$ for some $p \in (0, 1)$. Let $\ens[b] = \alpha \ens_1 + \bar{\alpha} \ens_2$ with $0 \leq \alpha < p$. As shown in the figure, suppose $\ens[b]$ is not separate from $\ens$. Then we can find $\ens[c] \in \Ens$ such that $\ens[b] = \beta \ens[c] + \bar{\beta} \ens[d]$ and $\ens = \gamma \ens[c] + \bar{\gamma} \ens[f]$ for some $\ens[d], \ens[f] \in \Ens$ and $\beta, \gamma \in (0, 1)$.
	
	Setting $\epsilon = \frac{p - \alpha}{\bar{\alpha}}$ and $\lambda = \bar{\epsilon} \beta$ we have:
	\begin{align*}
		\ens[a] &= p \ens_1 + \bar{p} \ens_2 = \left(p - \frac{\bar{p}}{\bar{\alpha}} \alpha \right) \ens_1 + \frac{\bar{p}}{\bar{\alpha}} \alpha \ens_1 + \frac{\bar{p}}{\bar{\alpha}} \bar{\alpha}\ens_2 \\
		&= \left(\frac{p\bar{\alpha} - \bar{p}\alpha}{\bar{\alpha}} \right) \ens_1 + \frac{\bar{p}}{\bar{\alpha}} (\alpha \ens_1 + \bar{\alpha} \ens_2) = \left(\frac{p - p\alpha - \alpha + p \alpha}{\bar{\alpha}} \right) \ens_1 + \frac{1 - p + \alpha - \alpha}{\bar{\alpha}} (\alpha \ens_1 + \bar{\alpha} \ens_2) \\
		&= \frac{p - \alpha}{\bar{\alpha}}  \ens_1 + \left( 1 - \frac{p - \alpha}{\bar{\alpha}}\right) (\alpha \ens_1 + \bar{\alpha} \ens_2) = \epsilon \ens_1 + \bar{\epsilon} (\alpha \ens_1 + \bar{\alpha} \ens_2) = \epsilon \ens_1 + \bar{\epsilon} \ens[b] \\
		&= \epsilon \ens_1 + \bar{\epsilon} ( \beta \ens[c] + \bar{\beta} \ens[d] ) = \bar{\epsilon} \beta \ens[c] + \epsilon \ens_1 + \bar{\epsilon} \bar{\beta} \ens[d] = \lambda \ens[c] + \bar{\lambda} \ens[g]
	\end{align*}
	where $\ens[g] = \frac{1}{\bar{\lambda}}\left( \epsilon \ens_1 + \bar{\epsilon} \bar{\beta} \ens[d] \right)$. This means $\ens[a]$ and $\ens$ have a common component, which is a contradiction. Therefore $\ens \separate \alpha \ens_1 + \bar{\alpha} \ens_2$ for all $\alpha \in [0, p]$.
	
	We can repeat the argument switching $\ens_1$ with $\ens_2$ and find $\ens \separate \alpha \ens_1 + \bar{\alpha} \ens_2$ for all $\alpha \in [0, 1]$.
\end{proof}

\end{mathSection}

While separateness extends to mixtures, the converse property (i.e. mixtures preserve separateness) is not necessarily true. While it is true for classical spaces, it is not true for quantum spaces. This converse property is, in effect, a telltale of classicality.

\begin{mathSection}
\begin{defn}
	Let $\Ens$ be an ensemble space. We say that \textbf{mixtures preserve separateness in $\Ens$} if $\ens \separate \ens[a]$ and $\ens \separate \ens[b]$ implies $\ens \separate p \ens[a] + \bar{p} \ens[b]$ for all $p \in [0,1]$ and $\ens, \ens[a], \ens[b] \in \Ens$.
\end{defn}

\begin{prop}
	Mixtures preserve separateness in discrete/continuous classical ensemble spaces.
\end{prop}

\begin{proof}
	Let $\Ens$ be a discrete or continuous classical ensemble space. Let $\ens, \ens[a], \ens[b] \in \Ens$ such that $\ens \separate \ens[a]$ and $\ens \separate \ens[b]$. Then the support of $\ens$ is disjoint from the support of both $\ens[a]$ and $\ens[b]$. Since the support of a mixture of $\ens[a]$ and $\ens[b]$ is the union of the supports, $\ens$ has disjoint support from every mixture of $\ens[a]$ and $\ens[b]$. This means that mixtures preserve separateness in discrete/continuous classical ensemble spaces.
\end{proof}

\begin{prop}
	Mixtures do not preserve separateness in quantum ensemble spaces
\end{prop}

\begin{figure}[H]
	\centering
	\begin{tikzpicture}
		%\draw [help lines] (0,0) grid (4,4);
		
		\coordinate (center) at (2,2);
		\def\radius{1.5}
		\coordinate (b) at ($(center)+\radius*({cos(95)},{sin(95)})$);
		\coordinate (a) at ($(center)+\radius*({cos(-85)},{sin(-85)})$);
		\coordinate (e) at ($(center)+\radius*({cos(155)},{sin(155)})$);
		\coordinate (f) at ($(center)+\radius*({cos(-25)},{sin(-25)})$);
		\coordinate (o) at ($(a)!0.5!(b)$);
		\coordinate (oa) at ($(a)!-0.2!(e)$);
		\coordinate (ob) at ($(b)!-0.5!(e)$);
		\draw [fill=ensembleFill, thick] (center) circle (\radius);
		\filldraw [black] (a) circle (1pt) node[below left] {$\ens[a]$};
		\filldraw [black] (b) circle (1pt) node[above left] {$\ens[b]$};
		\filldraw [black] (e) circle (1pt) node[left] {$\ens$};
		\filldraw [black] (o) circle (1pt) node[above right] {$\ens[o]$};
		\filldraw [black] (f) circle (1pt);
		\draw[] (e) -- (a) -- (b) -- (e) -- (f);
		\draw[dashed] (b) -- (ob);
		\draw[dashed] (a) -- (oa);	
	\end{tikzpicture}
	%\includegraphics[width=0.3\textwidth]{tempimages/MixturesDoNotPreserveSeparateness.jpg}
\end{figure}

\begin{proof}
	Let $\Ens$ be a quantum ensemble space. As shown in the figure, let $\ens[a], \ens[b] \in \Ens$ be two orthogonal pure states and let $\ens[e] \in \Ens$ be another pure state that is the nontrivial superposition of the two. These three states will be extreme points of a Bloch ball. Since they are all pure states, they are all extreme points and therefore are pairwise separate. Consider $\ens[o] = \frac{1}{2} \ens[a] + \frac{1}{2} \ens[b]$. This will be the center of the Bloch ball and will not be separate from $\ens$. Therefore mixtures do not preserve separateness in $\Ens$.
\end{proof}
\end{mathSection}

\subsection{Decomposability}

Since mixture preserving separateness is a telltale of classicality, it is insightful to find an alternative characterization. One of the differences between classical and quantum ensemble spaces is that quantum ensembles allow multiple decomposition in terms of pure states. We see here that, in fact, the lack of multiple decomposition is equivalent to mixture preserving separateness.

Note that, since a continuous classical ensemble space has no extreme points, we have to find a definition of multiple decomposability that makes no reference to the extreme points. Like for the definition of the entropy, we need to find a definition on mixtures of two elements that, when applied recursively, gives us the desired effect. The idea here is that we will always have multiple decompositions in terms of other ensembles, but in classical spaces we cannot have multiple decompositions where one element of the first decomposition has no common components with all elements of the second. So, if we start breaking an ensemble into separate components, while we may take different paths, we will reach the same final decomposition in terms of extreme points, if they exist.\footnote{There is an open issue as it is not clear whether we want to require actual separate decomposition or separate decomposition in the limit. In the continuous classical case, for example, it is not clear whether we should require probability densities to be continuous or not. If continuity is required, a probability density cannot be split into two probability densities with disjoint support without creating a discontinuity. However, we may think of a sequence of decomposition into three, one with support $A$, one with support $B$ and one with support on both. In the limit, the mixing coefficient for the third becomes smaller and smaller, meaning that the first two approach a discontinuous distribution.}

\begin{mathSection}
\begin{defn}
	An ensemble is \textbf{decomposable} if it can be expressed as a mixture of two distinct ensembles. An ensemble is \textbf{separately decomposable} if it can be expressed as a mixture of two separate ensembles. An ensemble is \textbf{multidecomposable} if it can be expressed as two decompositions where a component of one is separate from both components of the other. That is, $\ens = p \ens[a]_1 + \bar{p} \ens[a]_2 = \lambda \ens[b]_1 +\bar{\lambda} \ens[b]_2$ and either $\ens[a]_1 \separate \ens[b]_j$ or $\ens[a]_2 \separate \ens[b]_j$. An ensemble is \textbf{monodecomposable} if it is not multidecomposable. An ensemble is \textbf{separately monodecomposable} if it is both separately decomposable and monodecomposable.
\end{defn}

\begin{figure}[H]
	\centering
	\begin{tikzpicture}		
		%\draw [help lines] (0,0) grid (10,3);
		
		% Equilater triangle
		\coordinate (center) at (1.5,1.2);
		\def\angle{90}
		\def\radius{1.5}
		\coordinate (e1) at ($(center)+\radius*({cos(0+\angle)},{sin(0+\angle)})$);
		\coordinate (e2) at ($(center)+\radius*({cos(120+\angle)},{sin(120+\angle)})$);
		\coordinate (e3) at ($(center)+\radius*({cos(240+\angle)},{sin(240+\angle)})$);
		\coordinate (a) at ($(e2)!0.7!(e3)$);
		\coordinate (b) at ($(e1)!0.7!(e3)$);
		\coordinate (e) at ($(e1)!0.768!(a)$);
		\draw[fill=ensembleFill, thick] (e1) -- (e2) -- (e3) -- cycle;
		\filldraw [black] (e1) circle (1pt) node[above] {$\ens_1$};
		\filldraw [black] (e2) circle (1pt) node[below left] {$\ens_2$};
		\filldraw [black] (e3) circle (1pt) node[below right] {$\ens_3$};
		\filldraw [black] (a) circle (1pt);
		\filldraw [black] (b) circle (1pt);
		\filldraw [black] (e) circle (1pt);
		\draw (e1) -- (a);
		\draw (e2) -- (b);
		
		% Cut triangle
		\coordinate (center) at (6,1.2);
		\def\angle{90}
		\def\radius{1.5}
		\coordinate (t) at ($(center)+\radius*({cos(0+\angle)},{sin(0+\angle)})$);
		\coordinate (e2) at ($(center)+\radius*({cos(120+\angle)},{sin(120+\angle)})$);
		\coordinate (e3) at ($(center)+\radius*({cos(240+\angle)},{sin(240+\angle)})$);
		\coordinate (e1) at ($(e2)!0.5!(t)$);
		\coordinate (e4) at ($(e3)!0.5!(t)$);
		\coordinate (a) at ($(e2)!0.7!(e3)$);
		\coordinate (e) at ($(e1)!0.412!(a)$);
		\draw[fill=ensembleFill, thick] (e1) -- (e2) -- (e3) -- (e4) -- cycle;
		\filldraw [black] (e1) circle (1pt) node[above left] {$\ens_1$};
		\filldraw [black] (e2) circle (1pt) node[below left] {$\ens_2$};
		\filldraw [black] (e3) circle (1pt) node[below right] {$\ens_3$};
		\filldraw [black] (e4) circle (1pt) node[above right] {$\ens_4$};
		\filldraw [black] (a) circle (1pt);
		\filldraw [black] (e) circle (1pt);
		\draw (e1) -- (a);
		\draw (e2) -- (e4);
		
		
		\coordinate (center) at (10,1.5);
		\def\radius{1.25}
		\coordinate (b) at ($(center)+\radius*({cos(95)},{sin(95)})$);
		\coordinate (a) at ($(center)+\radius*({cos(-85)},{sin(-85)})$);
		\coordinate (e) at ($(center)+\radius*({cos(155)},{sin(155)})$);
		\coordinate (f) at ($(center)+\radius*({cos(-25)},{sin(-25)})$);
		\coordinate (o) at ($(a)!0.5!(b)$);
		\draw [fill=ensembleFill, thick] (center) circle (\radius);
		\filldraw [black] (a) circle (1pt);
		\filldraw [black] (b) circle (1pt);
		\filldraw [black] (e) circle (1pt);
		\filldraw [black] (o) circle (1pt);
		\filldraw [black] (f) circle (1pt);
		\draw[] (e) -- (f);
		\draw[] (a) -- (b);
	\end{tikzpicture}
	\caption{Examples for mono- and multidecomposability.}\label{pm-es-fig-monodecomposability}
	%\includegraphics[width=0.4\textwidth]{tempimages/MultipleDecomposition.jpg}
\end{figure}

\begin{remark}
	As shown in Fig. \ref{pm-es-fig-monodecomposability}, take a classical discrete space for three points which is a triangle (simplex). The only three elements that are not decomposable are the extreme points. Mixtures of two points are decomposable and are also separately decomposable in only one way. Mixtures of three points are also separately decomposable, but in multiple ways: as a mixture of $\ens_1$ and a mixture of $\ens_2$ and $\ens_3$, or as a mixture of $\ens_2$ and a mixture of $\ens_1$ and $\ens_3$. Note, however, that they are not separately multidecomposable as the different components are not separate.
	
	Take a Bloch ball for quantum mechanics. All the elements of the surface are not decomposable and they are all pairwise separate. The middle point can be seen as the equal mixture of any pair of opposite points. Therefore the middle point, as well as any other point not on the surface, is not only separately decomposable but also multidecomposable.
	
	To see why we require only one component to be separate from the other two, consider the cut triangle. Here we can have multiple decompositions where not all elements are separate.
\end{remark}

\begin{coro}
	An ensemble $\ens \in \Ens$ is an extreme point if and only if it not decomposable.
\end{coro}

\begin{proof}
	If $\ens \in \Ens$ is decomposable, then it has at least two distinct components, one of which must not be $\ens$. Since an extreme point has only itself as a component, then $\ens$ is not an extreme point. Conversely, if $\ens$ is an extreme point, it cannot be the mixture of two distinct components as $\ens$ has only itself as a component.
\end{proof}

\begin{defn}
	An ensemble space is \textbf{separately decomposable} if every decomposable ensemble is separately decomposable, \textbf{multidecomposable} if every decomposable ensemble is multidecomposable, \textbf{monodecomposable} if every decomposable ensemble is monodecomposable and \textbf{separately monodecomposable} if it is both separately decomposable and monodecomposable.
\end{defn}

\begin{prop}
	Discrete/continuous classical and quantum ensemble spaces are separately decomposable.
\end{prop}

\begin{proof}
	For a classical ensemble space, only Dirac measures are not decomposable. For a discrete classical space, every measure that is not the Dirac measure is the mixture of two measures with disjoint support. For a continuous classical space, there are no extreme points, and any absolutely continuous measure is the mixture of two absolutely continuous measures with disjoint support. Since measures with disjoint support are separate, every decomposable ensemble is separately decomposable.
	
	In quantum mechanics, every density operator is the mixture of its eigenstates. If the density operator does not correspond to a pure state, it will have at least two eigenstates. Every mixed state, then, is a mixture of two orthogonal ensembles. Since orthogonal ensembles cannot have a common component, every decomposable ensemble is separately decomposable.
\end{proof}

\begin{prop}
	Given an ensemble space $\Ens$, $\Ens$ is monodecomposable if and only if mixtures preserve separateness in $\Ens$.
\end{prop}

\begin{figure}[H]
	\centering
	\begin{tikzpicture}
		%\draw [help lines] (0,0) grid (4,4);
		
		\coordinate (e) at (1.6,0.5);
		\coordinate (a) at (0.2,2.5);
		\coordinate (b) at (3.0,2.5);
		\coordinate (c) at ($(a)!0.4!(b)$);
		\coordinate (d) at (0.3,0.4);
		\coordinate (g) at ($(e)!-0.8!(d)$);
		\coordinate (f) at ($(c)!-0.4!(d)$);
		\filldraw [black] (e) circle (1pt) node[below] {$\ens$};
		\filldraw [black] (a) circle (1pt) node[left] {$\ens[a]$};
		\filldraw [black] (b) circle (1pt) node[right] {$\ens[b]$};
		\filldraw [black] (c) circle (1pt) node[above left] {$\ens[c]$};
		\filldraw [black] (d) circle (1pt) node[left] {$\ens[d]$};
		\filldraw [black] (f) circle (1pt) node[right] {$\ens[f]$};
		\filldraw [black] (g) circle (1pt) node[right] {$\ens[g]$};
		\draw[thick] (a) -- (b) -- (e) -- cycle;
		\draw[thick] (g) -- (d) -- (f);
	\end{tikzpicture}
	%\includegraphics[width=0.3\textwidth]{tempimages/MonodecomposabilityIsMixturePreserveSeparateness.jpg}
\end{figure}

\begin{proof}
	Suppose mixtures do not preserve separateness in $\Ens$. Then, as shown in the figure, we can find $\ens,\ens[a],\ens[b], \ens[c] \in \Ens$ such that $\ens \separate \ens[a]$, $\ens \separate \ens[b]$ and $\ens \nseparate \ens[c] = p \ens[a] + \bar{p} \ens[b]$ for some $p \in (0,1)$. Since $\ens \nseparate \ens[c]$, we can find $\ens[d], \ens[f], \ens[g]$ such that $\ens[c] = \lambda \ens[d] + \bar{\lambda} \ens[f]$ and $\ens = \mu \ens[d] + \bar{\mu} \ens[g]$.  Since separateness extends to all mixtures (\ref{pm-es-separateExtendsMixtures}) and $\ens[a] \separate \ens[e] = \mu \ens[d] + \bar{\mu} \ens[g]$, then $\ens[a] \separate \ens[d]$. Similarly, $\ens[b] \separate \ens[d]$, which means that $\ens[c]$ is separately multidecomposable. Therefore, if mixtures do not preserve separateness in $\Ens$, not all decomposable ensembles are monodecomposable which means $\Ens$ is not monodecomposable.
	
	Now suppose mixtures do preserve separateness, and let $\ens = p \ens[a]_1 + \bar{p} \ens[a]_2 = \lambda \ens[b]_1 +\bar{\lambda} \ens[b]_2$ be a decomposable ensemble. Since $\ens[a]_1$ is a component of $\ens$, then $\ens \nseparate \ens[a]_1$. Since $\ens$ is a mixture of $\ens[b]_1$ and $\ens[b]_2$ and mixtures preserve separateness, then either $\ens[a]_1 \nseparate \ens[b]_1$ or $\ens[a]_1 \nseparate \ens[b]_2$. Similarly, $\ens[a]_2 \nseparate \ens[b]_1$ or $\ens[a]_2 \nseparate \ens[b]_2$. Therefore $\ens$ is not multidecomposable. Since this applies to all decomposable ensembles $\ens$, $\Ens$ is monodecomposable.
\end{proof}
\end{mathSection}

These definitions may be enough to prove that every finite-dimensional separately monodecomposable convex space is a simplex. For the infinite case, it would be nice to compare this characterization to a Choquet simplex.

\begin{conj}
	A finite-dimensional (i.e. there is a set of finitely many elements whose hull has non-empty interior) separately monodecomposable convex space is a simplex.
\end{conj}

\subsection{Convex subsets and convex hull}

In many cases, we will need to discuss the sets that contain all their possible mixtures. One typically distinguishes two cases. A set is convex if it allows all possible finite mixtures. This may be too restrictive as it may not include all possible infinite mixtures. A set is closed and convex if it includes all finite mixtures and their topological limits. Given that infinite mixtures are the topological limit of finite mixtures, a closed convex set contains all infinite mixtures. However, not all topological limits can be expressed as infinite mixture. For example, on the real line $1$ can be seen as a limit of points within the open interval $(0,1)$, but not as infinite convex combination. Therefore we add the notion of $\sigma$-convex set, a set that is closed under infinite mixtures.\footnote{The mathematical properties of $\sigma$-convex sets are yet to be explored.}

\begin{mathSection}
\begin{defn}
	Let $\Ens$ be an ensemble space. We say $A \subseteq \Ens$ is \textbf{convex} if it closed under finite mixtures (i.e. $\ens[a],\ens[b] \in A$ implies $p\ens[a] + \bar{p} \ens[b] \in A$ with $p \in [0,1]$), \textbf{$\sigma$-convex} if it is closed under infinite mixtures (i.e. $\ens[a]_i \in A$ implies $\sum_i p_i \ens[a]_i \in A$ for all possible infinite mixtures) and \textbf{closed and convex} if it is both convex and topologically closed.
\end{defn}

\begin{coro}
	A closed and convex set is $\sigma$-convex. A $\sigma$-convex set is convex.
\end{coro}
\end{mathSection}

Given a set of ensembles $A$, we can ask for all ensembles that can be constructed from $A$. The hull of $A$ is the set of all finite mixtures of $A$, the $\sigma$-hull of $A$ is the set of all infinite mixtures of $A$ and the closed hull of $A$ is the set of all the topological limits of finite mixtures of $A$. Notably, the closed hull of $A$ is equivalent to the topological closure of the hull of $A$.

\begin{mathSection}
\begin{defn}
	Let $A \subseteq \Ens$ be a subset of an ensemble space. The \textbf{convex hull} of $A$, noted $\hull(A)$ is the set of all finite mixtures of elements contained in $A$ (i.e. it is the smallest convex set that contains $A$). The \textbf{$\sigma$-hull} of $A$, noted $\shull(A)$ is the set of all infinite mixtures of elements contained in $A$ (i.e. it is the smallest $\sigma$-convex set that contains $A$). The \textbf{closed hull} of $A$, noted $\chull(A)$ is the smallest closed convex set that contains $A$. 
\end{defn}

\begin{remark}
	Note that, given a set $A$, not all elements of $\chull(A)$ can be understood as infinite mixtures. That is, we can have $\shull(A) \subset \chull(A)$. For example, let $\Ens$ be the line segment $[0,1]$ and consider the set $A=\left\{\frac{1}{2^i}\right\}_{i=0}^{\infty}$. Every point in $(0,1]$ can be expressed as a finite mixture of two elements of $A$, for example, $1$ and any number smaller than the target number. However, zero cannot be expressed as a convex combination of positive numbers, and therefore it is not an infinite mixture of $A$. However, zero is the limit of the sequence, and therefore it will be in the topological closure of $A$. This shows that the difference between $\sigma$-hull and convex hull exists already in finite dimensions. The convex hull and the $\sigma$-hull, instead, are the same in finite dimensions because \href{https://en.wikipedia.org/wiki/Carath%C3%A9odory%27s_theorem_%28convex_hull%29}{Carathéodory's theorem} allows us to rewrite any infinite convex combination into a finite one.
	
	For an example in which all hulls are different, consider the space of probability distributions $\Ens$ over countably many elements $X = \{x_i\}_{i=1}^{\infty}$. Let $\ens[a]_{ip} = p x_i + \bar{p} x_{i+1}$ and let $A = \left\{ \ens[a]_{ip}  \, | \, i \geq 1, p \in (0,1)  \right\} \subset \Ens$ be the set of all non-trivial mixtures of pairs of consecutive elements. A probability distribution with support over the full $X$ cannot be expressed as a finite convex combination of elements of $A$, and will therefore not be in the convex hull. However, it can be expressed as an infinite convex combination, and therefore it will be in the $\sigma$-hull. An element $x_i \in X$, is not in the $\sigma$-hull, but it will be in the closed hull, as $x_i = \lim_{p \to 1} p x_i + \bar{p} x_{i+1} = \lim_{p \to 1} \ens[a]_{ip} \in \chull(A)$.
\end{remark}

\begin{coro}
	Given $A \subseteq \Ens$, $\hull(A) \subseteq \shull(A) \subseteq \chull(A)$.
\end{coro}

\begin{proof}
	All finite mixtures are also infinite mixtures with $p_i = 0$ for all $i > n$ for some $n$. Therefore $\hull(A) \subseteq \shull(A)$. All infinite mixtures are topological limits of finite mixtures. Therefore $\shull(A) \subseteq \chull(A)$. 
\end{proof}

\begin{prop}\label{pm-es-hullProp}
	All three hull operators are closures. That is, $\hull$ satisfies the following three properties:
	\begin{enumerate}
		\item \textbf{extensive}: $A \subseteq \hull(A)$
		\item \textbf{increasing}: $A \subseteq B \implies \hull(A) \subseteq \hull(B)$
		\item \textbf{idempotent}: $\hull(\hull(A)) = \hull(A)$
	\end{enumerate}
	and similarly do $\shull$ and $\chull$.
\end{prop}

\begin{proof}
	1. Every element of $A$ is trivially a mixture of elements of $A$. Therefore $A \subseteq \hull(A)$. Since $\hull(A) \subseteq \shull(A) \subseteq \chull(A)$, $A \subseteq \shull(A)$ and $A \subseteq \chull(A)$ as well. 
	
	2. Let $\ens \in \hull(A)$. Then it is a finite mixture of some elements of $A$. Since $A \subseteq B$, then $\ens$ is also the finite mixture of some elements of $B$ and therefore $\ens \in \hull(B)$. The same logic applies to the $\sigma$-hull and closed hull replacing finite mixture with the appropriate operation.
	
	3. Since $\hull(\hull(A))$ is the smallest convex subset that contains $\hull(A)$, and since $\hull(A)$ is a convex subset, then $\hull(\hull(A))$ must be $\hull(A)$ since no smaller set can contain all elements of $\hull(A)$. The same logic applies to the $\sigma$-hull and closed hull.
\end{proof}

\begin{coro}
	A subset $A \subseteq \Ens$ is respectively convex/$\sigma$-convex/closed convex if and only if it is its own convex hull/$\sigma$-hull/closed hull.
\end{coro}

\begin{proof}
	Let $A \subseteq \Ens$ be a convex subset. By \ref{pm-es-hullProp} we have $A \subseteq \hull(A)$. By definition of convex set, we have $\hull(A) \subseteq A$. Therefore $A = \hull(A)$. Conversely, let $A \subseteq \Ens$ be a set of ensembles not necessarily convex and let $A=\hull(A)$. By definition, $\hull(A)$ is closed under finite mixture and is therefore a convex subset. The same logic applies to $\sigma$-convex and closed convex sets with the respective hulls.
\end{proof}

\begin{defn}
	We note $\mathfrak{co}_{\Ens}$ the set of all convex subsets of $\Ens$, $\mathfrak{sco}_{\Ens}$ the set of all $\sigma$-convex subsets of $\Ens$ and $\mathfrak{cco}_{\Ens}$ the set of all closed convex subsets of $\Ens$.
\end{defn}

\begin{prop}
	The sets $\mathfrak{co}_{\Ens}$, $\mathfrak{sco}_{\Ens}$ and $\mathfrak{cco}_{\Ens}$, as posets ordered by inclusion, are topped $\bigcap$-structures and therefore complete lattices.
\end{prop}

\begin{proof}
	Theorem 7.3 in Davey and Priestley's ``Introduction to Lattice and Order'' states that, given a closure operator, the set of all closures, ordered by inclusion, is a topped $\bigcap$-structure and, therefore, a complete lattice. Since $\mathfrak{co}_{\Ens}$, $\mathfrak{sco}_{\Ens}$ and $\mathfrak{cco}_{\Ens}$ are closures, the theorem applies.
\end{proof}


\begin{prop}
	The functions $\hull$, $\shull$ and $\chull$ are continuous from above. That is, given a decreasing sequence $A_i \subseteq \Ens$, $\hull(\lim\limits_{i \to \infty} A_i) = \lim\limits_{i \to \infty} \hull(A_i)$. Similarly for $\shull$ and $\chull$.
\end{prop}

\begin{proof}
	The above proposition is a consequence of the fact that the hulls are closure operations and they generate an intersection structure. This means that the intersection of hulls is the hull of the intersections.
	
	Let $A_i \subseteq \Ens$ be a decreasing sequence. That is, $A_{i+1} \subseteq A_i$. Then $A = \lim\limits_{i \to \infty} A_i = \bigcap A_i$. Since $\hull$ is order preserving, $\hull(A_i)$ is a decreasing sequence and $\lim\limits_{i \to \infty} \hull(A_i) = \bigcap \hull(A_i)$. Moreover, $\hull(A) \subseteq \hull(A_i)$ for all $i$ and therefore $\hull(A) \subseteq \bigcap \hull(A_i)$. Now let $\ens \in \hull(A)$. Then $\ens$ is a convex combination of elements of $A$. Since every element of $A$ is also an element of any $A_i$, then $\ens$ is also a convex combination of elements of $A_i$ for any $i$. Therefore $\ens \in \hull(A_i)$ for all $i$ which means $\ens \in \bigcap \hull(A_i)$ and therefore $\hull(A) \supseteq \bigcap \hull(A_i)$. Thus we have that $\hull(\lim\limits_{i \to \infty} A_i) = \lim\limits_{i \to \infty} \hull(A_i)$.
	
	Since we have only used closure properties of $\hull$, the same reasoning applies to $\shull$ and $\chull$ since they are closures.
\end{proof}

\begin{prop}
	The $\hull$ is continuous from below. That is, let $A_i \subseteq \Ens$ be an increasing sequence. Then $\hull(\lim\limits_{i \to \infty} A_i) = \lim\limits_{i \to \infty} \hull(A_i)$.
\end{prop}

\begin{proof}
	Let $A_i \subseteq \Ens$ be an increasing sequence. That is, $A_{i+1} \supseteq A_i$. Then $A = \lim\limits_{i \to \infty} A_i = \bigcup A_i$. Since $\hull$ is an increasing function, $\hull(A) \supseteq \hull(A_i)$ for all $i$ and therefore $\hull(A) \supseteq \bigcup \hull(A_i)$. Now let $\ens \in \hull(A)$. Then $\ens$ is a convex combination of finitely many elements $\ens[a]_j$ of $A$. Since $A$ is the union of all $A_i$, each $\ens[a]_j$ will be in some $A_i$. Since the sequence of $A_i$ is increasing, and there are only finitely many $\ens[a]_j$, we will find an $i$ such that $\ens[a]_j \in A_i$ for all $j$. This means that $\ens$ is a convex combination of elements of $A_i$ and therefore $\ens \in \hull(A_i) \subseteq \hull(A)$. Therefore $\hull(A) = \bigcup \hull(A_i)$ which means $\hull(\lim\limits_{i \to \infty} A_i) = \lim\limits_{i \to \infty} \hull(A_i)$.
\end{proof}


\begin{remark}
	Note that $\shull$ and $\chull$ are not, in general, continuous from below. This is because, in general, the union of closures is not the closure of the union. This is true, in particular, with topological closures, which is part of the definition of $\chull$.
	
	For example, consider the sequence $A_i = \left[0, 1-\frac{1}{i}\right) \subseteq \mathbb{R}$. These are convex sets therefore their closed hull is simply their topological closure. That is, $\chull(A_i) = \left[0, 1-\frac{1}{i}\right]$. We have $\lim\limits_{i \to \infty} A_i = \bigcup A_i = \left[0, 1\right)$ and $\lim\limits_{i \to \infty} \chull(A_i) = \bigcup \chull(A_i) = \left[0, 1\right)$ which is not a closed set and therefore different from $\chull(A) = [0,1]$. The closed hull of the limit is not the limit of the convex hull, even in a finite-dimensional space.
	
	For the $\sigma$-hull, we need an infinite-dimensional example. Conceptually, we are using the fact that a uniform distribution over the whole $[0,1]$ is the infinite convex combination of uniform distributions over countably many sets that cover the whole $[0,1]$. Let $\Ens$ be the space of probability measures defined over $[0,1] \subseteq \mathbb{R}$. Let $\{\ens[a]_i\}_{i=1}^{\infty}$ be the sequence of uniform distributions over $\left[\frac{1}{i+1}, \frac{1}{i}\right]$. Let $\ens$ be the uniform distribution over $[0,1]$. We have $\ens = \sum \frac{1}{i(i+1)} \ens[a]_i$ where $\sum \frac{1}{i(i+1)} = 1$. Therefore $\ens$ is the countable convex combination of $\ens[a]_i$. Let $A_j = \{ \ens[a]_i \, | \, i \leq j\}$. Note that $\ens[a]_i \separate \ens[a]_j$ for all $i \neq j$. Therefore, $\{\ens[a]_i\}$ are exactly all extreme points of $\shull(\bigcup A_j)$. This means that $\ens \notin \shull(A_j)$ for all $j$ while $\ens \in \shull(\bigcup A_j)$. Therefore the $\sigma$-hull of the limit is not the limit of the $\sigma$-hulls.
\end{remark}


\begin{prop}
	The topological closure of the hull is a convex set and, therefore, the closed hull.
\end{prop}

\begin{proof}
	Let $A$ be a convex set and $\bar{A}$ its topological closure. Let $\ens[a]_i, \ens[b]_i \in A$ be two sequences that converge to $\ens[a], \ens[b] \in \Ens$ respectively. We have $\ens[a], \ens[b] \in \bar{A}$ since they are topological limits. Consider $\ens_i = p \ens[a]_i + \bar{p} \ens[b]_i$. Since mixing is continuous, we have:
	\begin{equation}
		\begin{aligned}
			\ens &= p \ens[a] + \bar{p} \ens[b] = p \lim\limits_{i \to \infty} \ens[a]_i + \bar{p} \lim\limits_{i \to \infty} \ens[b]_i = \lim\limits_{i \to \infty} (p \ens[a]_i + \bar{p} \ens[b]_i) = \lim\limits_{i \to \infty} \ens_i.
		\end{aligned}
	\end{equation}
	But $\ens_i$ are finite mixtures of elements of $A$, and therefore $\ens_i \in A$ is a sequence of elements of $A$. The sequence converges, $\ens$ is the limit of a sequence of elements of $A$ and therefore $\ens \in \bar{A}$. That is, the topological closure of a convex set is also a convex set. But this means that $\bar{A}$ is a closed convex set. Since any closed convex set that contains $A$ will also need to contain $\bar{A}$, $\bar{A}$ is the closed hull of $A$.

	Let $A \subset \Ens$ be a subset not necessarily convex. Then $\hull(A)$ will be convex. Therefore its topological closure will be the closed hull of $A$.
\end{proof}
\end{mathSection}

There may be a relationship between the algebraic notion of $\sigma$-convexity and the topological notion of interior. For example, let $U$ be a convex open set. While it clearly cannot be closed convex, is it $\sigma$-convex? The idea is that convexity can only return points that are ``inside'' the set, and $\sigma$-convexity is required to fill in all the limits. If that is true, it would be natural to look at some sort of converse. Clearly, not all $\sigma$-convex sets are open, since all closed convex sets are also $\sigma$-convex. The question becomes whether, for $\sigma$-convex sets, the notion of algebraic boundary and topological boundary coincides. These questions raise the following conjectures.

\begin{conj}
	Let $U \subseteq \Ens$ be a convex open set. Then $U$ is $\sigma$-convex.
\end{conj}

\begin{defn}
	Given a set $A \subset \Ens$, $\ens[a] \in A$ is an internal point of $A$ if for any $\ens \in \Ens$ we can find $\ens[b] \in A$ such that $\ens[a] = p \ens + \bar{p} \ens[b]$ for some $p \in (0,1]$.
\end{defn}

\begin{conj}
	 Let $A \subseteq \Ens$ and let $\ens[a] \in \chull(A)$ be an internal point of $\chull(A)$. Then $\ens[a] \in \shull(A)$.
\end{conj}

\begin{conj}
	Let $A \subseteq \Ens$ be a $\sigma$-convex set. Then $\ens[a] \in A$ is an internal point of $A$ if and only if it is an interior point of $A$.
\end{conj}

\begin{remark}
	Note that if $A$ is not convex, this is clearly not true. For example, let $A \subseteq \mathbb{R}^2$ be an annulus (i.e. the region of two concentric circles). The points on the inner circle are internal points of $A$ according to the definition, but they are not interior points. An internal point of $A$ for a $\sigma$-convex set is guaranteed to be surrounded by an open interval along any direction. The question, as usual, is if this is enough to fit an open set.
\end{remark}

\subsection{Convex supremum}

Later in the chapter, we will need to create the non-additive generalization for probability measures and state counting measures. It turns out that both constructions can be understood as instances of a more general construction that starts with a function $f(\ens)$ of elements of the ensemble space and generates a function $cs_f(A)$ of sets of ensembles by asking what is the highest value of $f$ that is reachable by a mixture of elements of $A$. We are going to show that this construction alone presents many nice mathematical properties.

\begin{mathSection}
\begin{defn}
	Given a function $f : \Ens \to \mathbb{R}$, a \textbf{convex supremum} of $f$ is a set function $cs_f : 2^{\Ens} \to [- \infty, + \infty]$ such that $cs_f(A) = \sup(f(\hull(A)) \cup \{f_{\emptyset}\})$, with $f_{\emptyset} \leq \inf(f(\Ens))$, that returns the highest value of $f$ reachable by convex combinations of $A$.
\end{defn}

\begin{prop}\label{pm-es-convexSupremumProps}
	For any $f$, the convex supremum $cs_f$ has the following properties
	\begin{enumerate}
		\item range of $f$: $cs_f(A) \in [f_{\emptyset}, \sup(f(\Ens))]$
		\item increasing: $A \subseteq B \implies cs_f(A) \leq cs_f(B)$
		\item continuous from below: for any increasing sequence $A_i \subseteq \Ens$, $cs_f(\lim\limits_{i \to \infty} A_i) = \lim\limits_{i \to \infty} cs_f(A_i)$.
	\end{enumerate}
\end{prop}

\begin{proof}
	1. If $A$ is empty, $cs_f(A) = f_{\emptyset}$. If $A$ is non-empty, $cs_f(A)$ is the supremum of the subset of values returned by $f(\hull(A))$, which means $f_{\emptyset} \leq \inf(f(\Ens)) \leq cs_f(A) \leq \sup(f(\Ens))$. Therefore $cs_f(A) \in [f_{\emptyset}, \sup(f(\Ens))]$ for all $A$.

	2. If $A = B = \emptyset$, then $cs_f(A) = cs_f(B)$. If $B \neq \emptyset$, $cs_f(A) = f_{\emptyset} \leq \inf(\Ens) \leq cs_f(B)$. In the last case, since the $\hull$ is an increasing function, the image of a set through a map is an increasing function and the supremum is an increasing function, the convex supremum is an increasing function.
	
	3. Let $A_i \subseteq \Ens$ be an increasing sequence and $A = \bigcup A_i$. Since $cs_f$ is increasing, $cs_f(A_i)$ is also an increasing sequence. Since $\hull$ is continuous from below, we have:
	\begin{equation}
		\begin{aligned}
			cs_f(A) &= \sup(f(\hull(A))\cup\{f_{\emptyset}\}) = \sup(f(\bigcup\hull(A_i))\cup\{f_{\emptyset}\})\\
			&= \sup(\bigcup f(\hull(A_i))\cup\{f_{\emptyset}\}) = \sup(\bigcup (\sup(f(\hull(A_i))\cup\{f_{\emptyset}\}))) \\
			&= \sup(\bigcup cs_f(A_i)).
		\end{aligned}
	\end{equation}
	But since $cs_f(A_i)$ is increasing, the supremum is exactly the limit. Therefore the convex supremum is continuous from below.
\end{proof}

%TODO: turn it into a theorem for a non-trivial ensemble space (i.e. assume existence of two distinct).

\begin{remark}
	Note that the convex supremum is not continuous from above. Let $\Ens$ be a disc embedded in $\mathbb{R}^2$. Let $f : \Ens \to \mathbb{R}$ be a non-trivial linear function. Then $f$ will have a maximum and a minimum on two opposite points $\ens[a]$ and $\ens[b]$ on the circle that encloses the disc. Take a line that divides the disc in two halves, leaving the minimum and the maximum on different halves. Then over that line $f$ will take the minimum value $m > f(\ens[b])$ on one of the extreme points of the line. Consider a countable collection $\{\ens_i\}$ of points over that line and let $A_j = \{ \ens_i \, | \, i \geq j \}$. This is a decreasing sequence of infinite sets, where we are taking one point out at a time.  We have $f(\ens_i) \geq m$ for all $i$ and therefore $cs_f(A_j) \geq m$ for all $j$, which means $\lim\limits_{j \to \infty} cs_f(A_j) \geq m$. However, $\bigcap A_j = \emptyset$ and $cs_f(\emptyset) = f_{\emptyset} \leq \inf(f(\Ens)) = f(\ens[b]) < m$. This means that, in general, $cs_f(\lim\limits_{j \to \infty} A_j) \neq \lim\limits_{j \to \infty} cs_f(A_j)$ for a decreasing sequence.
\end{remark}

\begin{prop}\label{pm-es-convexSupremumContinuous}
	Let $A \subseteq \Ens$ and $f: \Ens \to \mathbb{R}$ be a continuous function, then $cs_f(A) = cs_f(\shull(A)) = cs_f(\chull(A))$.
\end{prop}

\begin{proof}
	The proposition is true if $A = \emptyset$, since $\emptyset = \hull(\emptyset) = \shull(\emptyset) = \chull(\emptyset)$.
	
	Now let $A \neq \emptyset$. Note that $\chull(A)$ is a convex set, meaning $\hull(\chull(A)) = \chull(A)$. This means that we are looking for the difference between the supremum of $f$ over the hull and the closed hull. Suppose $\ens \in \chull(A)$ but $\ens \notin \hull(A)$. Still, $\ens$ will be the limit of a sequence of $\ens_i \in \hull(A)$. Since $f$ is continuous, then $f(\ens)$ is the limit of the sequence $f(\ens_i)$, for which $f(\ens_i) \leq \sup(f(\hull(A)))$ for all $i$. Therefore, $f(\ens) \leq \sup(f(\hull(A))) = cs_f(A)$. Since $f(\ens) \leq cs_f(A)$ for all $\ens \in \chull(A)$, then $\sup(f(\chull(A))) = cs_f(\chull(A)) \leq cs_f(A)$. Since $cs_f$ is increasing and $\hull(A) \subseteq \chull(A)$, $cs_f(A) \leq cs_f(\chull(A))$. Therefore $cs_f(A) = cs_f(\chull(A))$.
	
	Note that $A \subseteq \shull(A) \subseteq \chull(A)$ and $cs_f$ is an increasing function. Therefore $cs_f(A) \leq cs_f(\shull(A)) \leq cs_f(\chull(A)) = cs_f(A)$ which means $cs_f(A) = cs_f(\shull(A))$.
\end{proof}

\end{mathSection}

Note that a measure is a monotonic set function continuous from below that is also non-negative and additive. It is easy to show that the convex supremum is non-negative if and only if $f$ is non-negative. Additivity, instead, is more complicated as it needs to be recovered on the lattice of subspaces.

\begin{mathSection}
\begin{coro}
	A convex supremum of $f$ is non-negative if and only if $f$ is non-negative and $f_{\emptyset} \geq 0$.
\end{coro}
\begin{proof}
	Since the range of $cs_f$ is $[f_{\emptyset}, \sup(f(\Ens))]$, $cs_f$ is non-negative if and only if $f_{\emptyset} \geq 0$. Since $f_{\emptyset} \leq \inf(f(\Ens))$, $f_{\emptyset} \geq 0$ only if $f$ is non-negative.
\end{proof}
\end{mathSection}

\section{Axiom of entropy}

In this section we will see how characterizing the variability of the elements within an ensemble leads to the standard notion of entropy. The entropy will have to satisfy few basic properties justified by the notion of variability which will be enough to recover the usual Shannon formula. It will also provide us with another basic notion to compare ensembles: two ensembles are mutually exclusive, or orthogonal, if they have no elements in common, which means the entropy will maximally increase during mixing. The interaction between orthogonality and separateness is enough to understand the differences between classical and quantum systems.

We saw that our basic notion of state is that of an ensemble. Since an ensemble represents all possible preparations of equivalent systems prepared according to the same procedure, we want to characterize how much those instances are different from each other. To be truly general, we want to be able to make this characterization without assuming what the individual instances are. First of all, the variability should be an experimentally well-defined quantity, and it should therefore be a continuous function in the natural topology. It should also be compatible with statistical mixing. Intuitively, the variability cannot decrease during mixing which makes the entropy a strictly concave function. Variability will have a maximal increase when mixing ensembles that are mutually exclusive: no instance of one can be confused with an instance of the other. In that case, the increase of the variability is fully characterized by the mixing coefficients. We say that two mutually exclusive ensembles are orthogonal, as this property will correspond mathematically to orthogonality in the space of distributions. Lastly, if one ensemble has no instances in common with other two, then it has no instance in common with any mixture of the two.

Thinking of ensembles and the variability of their instances, then, makes it clear what entropy is and why it has these properties. We are not concerned, at this point, what the source of the variability is. We just know that it is something that we need to characterize.

\begin{mathSection}
\begin{axiom}[Axiom of entropy]\label{pm-es-axiomEntropy}
	Every ensemble is associated with an \textbf{entropy} which quantifies the variability of the instances within an ensemble. Formally, an ensemble space $\Ens$ is equipped with a function $S : \Ens \to \mathbb{R}$, defined up to a positive multiplicative constant representing the unit numerical value. The entropy has the following properties:
	\begin{itemize}
		\item \textbf{Continuity}\footnote{Currently, we are imposing that the entropy is continuous. There may be a chance that this requirement is redundant, as strict concavity and the upper variability bound may already impose this. We have found proofs that show that real-valued convex/concave functions of real values are continuous. These proofs fail at the extreme points, but the upper variability bound may fix this. Another open question is whether differentiability is also an independent requirement.}
		\item \textbf{Strict concavity}: $S(p\ens[a] + \bar{p} \ens[b]) \geq p S(\ens[a]) + \bar{p} S(\ens[b])$ with the equality holding if and only if $\ens[a] = \ens[b]$
		\item \textbf{Upper variability bound}: there exists a universal function $I(p, \bar{p})$ (i.e. the same for all ensemble spaces) such that $S(p\ens[a] + \bar{p} \ens[b]) \leq I(p, \bar{p}) + p S(\ens[a]) + \bar{p} S(\ens[b])$; if the equality holds, $\ens[a]$ and $\ens[b]$ are \textbf{mutually exclusive} or \textbf{orthogonal}, noted $\ens[a] \ortho \ens[b]$
		\item \textbf{Mixtures preserve orthogonality}:\footnote{It is unclear whether ``mixtures preserve orthogonality'' is an independent axiom. Intuitively, the following argument tells us that it is. Take the 2-dimensional simplex (i.e. a triangle) that represents a classical discrete probability space over three elements. Take the standard entropy, which will satisfy ``mixtures preserve orthogonality''. This is because the middle point has entropy $\log 3$. We can imagine redefining the entropy so that it is a little bit lower in the center but it is unchanged on the sides. However, one needs to provide an actual example and show that it satisfies all axioms. It may also be that only one direction of the implication is an independent axiom. That is, that orthogonality with the components implies orthogonality with the mixtures. This is what does not hold for separateness.} $\ens[a] \ortho \ens[b]$ and $\ens[a] \ortho \ens[c]$ if and only if $\ens[a] \ortho p \ens[b] + \bar{p} \ens[c]$ for any $p \in (0,1)$
	\end{itemize}
\end{axiom}

\begin{justification}
	The entropy quantifies the variability of the instances within an ensemble. Since the ensemble represents a collection of preparations of equivalent systems, and since each instance will in general be potentially different, it is legitimate to ask how much variability there is among the different instances. We are assuming that the entropy is a quantity (i.e. a linearly ordered property), meaning that it is always meaningful to tell whether one ensemble has more variability than another. If this is the case, the later requirements of continuity and strict concavity will force the entropy to be a real-valued quantity. This is because the variability will change under statistical mixtures, and since statistical mixtures are performed with real-valued coefficients, the variability will have to be a real-valued quantity. While we strongly suspect that it is not conceptually possible to have a characterization of variability that is not linearly ordered, as this would entail ranges of the same variable that can be potentially mapped to each other without a clear mapping of their variability, we do not yet have a tight argument. Therefore we are not able to fully justify entropy's linear ordering at this time, and the linear ordering of the entropy should be considered an assumption. Provided that assumption, we are justified to assume the existence of a real-valued function that returns the entropy, a measure of variability of the ensemble.
	
	Since the entropy is a real-valued quantity, it will have a corresponding unit. This unit is independent from all other units, and therefore the overall structure of the ensemble space must be independent of this choice. Mathematically, the physical dimension of the unit is not captured, just its numeric value. A change of unit may change the numeric value by a multiplicative constant. Since variability is an ordered quantity, we want the change of units to respect the ordering and therefore it should be a positive multiplicative constant. This justifies that the entropy function is defined up to a positive multiplicative constant.
	
	Note that the additivity of the entropy over independent systems fixes the zero. If $S_{AB} = S_A + S_B$, in fact, one can't rescale all three terms by an additive factor and preserve the relationship.
	
	The variability, in the end, will have physical consequences, and it will therefore be measurable, thus experimentally verifiable: it will have to be a topologically continuous function. Moreover, small changes in the ensemble should produce small changes in the variability, which justifies analytical continuity. We are therefore justified to assume continuity of the entropy.
	
	Suppose we have two ensembles and we perform a statistical mixture. There are going to be three sources of variability: the two ensembles and the random choice between them. The total contribution from the original ensembles will be the average variability of the original ensembles. This is increased by the variability introduced by the random choice, which is always a positive contribution. Therefore the final variability cannot be less than the average of the original ensembles. That is, $S(p\ens[a] + \bar{p} \ens[b]) \geq p S(\ens[a]) + \bar{p} S(\ens[b])$. If we are mixing an ensemble with itself, this is equivalent to just choosing from the original ensemble, therefore the variability will not increase. Conversely, if the variability stays the same, it means that the random choice does not increase the variability, and therefore we must be choosing between equivalent ensembles. Therefore we are justified to assume that entropy is strictly concave.
	
	On the other hand, the variability cannot increase arbitrarily during mixture. The maximum variability will be given when the two ensembles are mutually exclusive, when an instance of the first ensemble cannot be produced by the second ensemble. That is, a single instance is enough to determine whether we have the first ensemble or the second. In this case, the variability is increased by the variability of the random choice, which must depend only on the mixture coefficient, and not the nature of the ensembles themselves. That is, $S(p\ens[a] + \bar{p} \ens[b]) \leq I(p, \bar{p}) + p S(\ens[a]) + \bar{p} S(\ens[b])$ is the upper variability bound, which is saturated if and only if $\ens[a]$ and $\ens[b]$ are mutually exclusive. The actual function $I$ is left unspecified and, as we show in proposition \ref{pm-es-entropyUnique}, it will correspond to the Shannon entropy as it is the only indicator of variability that will satisfy the axiom of entropy. This justifies the upper variability bound.
	
	Now suppose ensembles $\ens[a]$ and $\ens[b]$ are mutually exclusive and so are $\ens[a]$ and $\ens[c]$. That is, an instance of $\ens[a]$ cannot ever be produced by either $\ens[b]$ or $\ens[c]$. Then an instance of $\ens[a]$ cannot be produced by a mixture of $\ens[b]$ and $\ens[c]$, since ultimately a mixture of $\ens[b]$ and $\ens[c]$ will return an instance of one of the two. Therefore $\ens[a]$ and any mixture of $\ens[b]$ and $\ens[c]$ are mutually exclusive. The argument works in reverse as well: if an instance of $\ens[a]$ cannot be produced by a mixture of $\ens[b]$ and $\ens[c]$, then it cannot be produced by either. This justifies mixtures preserve orthogonality.
\end{justification}
\end{mathSection}

As with the other axioms, we should now verify that the axiom of entropy is satisfied by the standard cases.

\begin{mathSection}
	\begin{prop}
		Discrete classical ensemble spaces, continuous classical ensemble spaces and quantum ensemble spaces satisfy the axiom of entropy.
	\end{prop}
	
	\begin{proof}
		Let's first look at the classical continuous case. Every ensemble is represented by a distribution $\rho(x)$ with $\int_X \rho(x) d\mu=1$. The entropy is given by $S(\rho) = - \int_X \rho \log \rho d\mu$. This is a continuous function of $\rho$.

		\begin{figure}[H]
			\centering
			\begin{tikzpicture}
			%\draw [help lines] (0,0) grid (4,4);
			\pgfplotsset{ticks=none}
				\begin{axis}[
					height=5.5cm,
					width=7cm,
					axis lines=middle,
					xlabel={$\rho$},
					xlabel style={below left},
					xmin=-1,xmax=4,
					ymin=-2,ymax=2
					]
					\addplot[black,samples=300]{-ln(x)/ln 2}
					node [pos=0.4, right] {$-\log \rho$};
					\addplot[black,samples=30] {(1-x)/ln(2)}
				    node [pos=0.7, left] {$\frac{1-\rho}{\ln 2}$};
				\end{axis}
			\end{tikzpicture}
			%\includegraphics[width=0.5\textwidth]{tempimages/LogBound.jpg}
		\end{figure}
		
		Recall that throughout this chapter, all logarithms are base two. To show strict concavity, note that, as shown in the figure, $- \log \rho \geq \frac{1-\rho}{\ln 2}$ with the equality holding if and only if $\rho=1$. We have
		\begin{equation}
			\begin{aligned}
				S(p\rho_1 + \bar{p}\rho_2) &= - \int_X \left(p\rho_1 + \bar{p}\rho_2\right) \log \left(p\rho_1 + \bar{p}\rho_2\right) d\mu \\
				&= - \int_X p\rho_1 \log \left(p\rho_1 + \bar{p}\rho_2\right) d\mu - \int_X \bar{p}\rho_2 \log \left(p\rho_1 + \bar{p}\rho_2\right) d\mu \\
				&= - \int_X p\rho_1 \log \frac{p\rho_1 + \bar{p}\rho_2}{\rho_1} d\mu - \int_X p\rho_1 \log \rho_1 d\mu \\
				&- \int_X \bar{p}\rho_2 \log \frac{p\rho_1 + \bar{p}\rho_2}{\rho_2} d\mu - \int_X \bar{p}\rho_2 \log \rho_2 d\mu \\
				&\geq \int_X p\rho_1 \frac{1}{\ln 2} \left(1 - \frac{p\rho_1 + \bar{p}\rho_2}{\rho_1} \right)   d\mu - p \int_X \rho_1 \log \rho_1 d\mu \\
				&+ \int_X \bar{p}\rho_2 \frac{1}{\ln 2} \left(1 - \frac{p\rho_1 + \bar{p}\rho_2}{\rho_2} \right)   d\mu - \bar{p} \int_X \rho_2 \log \rho_2 d\mu \\
				&= \frac{p}{\ln 2}\left[ \int_X \rho_1 d\mu - \int_X \left(p\rho_1 + \bar{p}\rho_2\right) d\mu  \right] + p S(\rho_1) \\
				&+ \frac{\bar{p}}{\ln 2}\left[ \int_X \rho_2 d\mu - \int_X \left(p\rho_1 + \bar{p}\rho_2\right) d\mu  \right] + \bar{p} S(\rho_2) \\
				&= \frac{p}{\ln 2}\left[ 1 - 1  \right] + p S(\rho_1) + \frac{\bar{p}}{\ln 2}\left[ 1 - 1 \right] + \bar{p} S(\rho_2) \\
				&= p S(\rho_1) + \bar{p} S(\rho_2) \\
			\end{aligned}
		\end{equation}
		The equality holds if and only if $\frac{p\rho_1 + \bar{p}\rho_2}{\rho_1} = 1$ which is exactly when $\rho_1 = \rho_2$.
		
		

		\begin{figure}[H]
			\centering
			\begin{tikzpicture}
				%\draw [help lines] (0,0) grid (4,4);
				\pgfplotsset{ticks=none}
				\begin{axis}[
					height=5.5cm,
					width=7cm,
					axis lines=middle,
					xlabel={$\rho$},
					xlabel style={below left},
					xmin=-1,xmax=4,
					ymin=-2,ymax=2
					]
					\addplot[black,samples=300]{-ln(x)/ln 2}
					node [pos=0.4, right] {$-\log \rho$};
					\addplot[dashed, gray] coordinates {(1.8,0) (1.8,-0.84799) (0,-0.84799)};
					\addplot[dashed, gray] coordinates {(2.2,0) (2.2,-1.1375) (0,-1.1375)};
				\end{axis}
				\node at (3,2) [above] {$\hat{\rho}$};
				\node at (3.3,2) [above right] {$\hat{\rho}+k$};
				\node at (1.1,1) [above left] {$\scriptstyle -\log(\hat{\rho})$};
				\node at (1.1,1) [below left] {$\scriptstyle -\log(\hat{\rho}+k)$};
			\end{tikzpicture}
			%\includegraphics[width=0.5\textwidth]{tempimages/LogMonotone.jpg}
		\end{figure}
		For the upper bound, as shown in the figure, note that the logarithm is a strictly increasing function, and therefore $- \log(\hat{\rho} + k) \leq -\log(\hat{\rho})$ for any $k \geq 0$, with equality holding if and only if $k=0$. We have
		\begin{equation}
			\begin{aligned}
				S(p\rho_1 + \bar{p}\rho_2) &= - \int_X \left(p\rho_1 + \bar{p}\rho_2\right) \log \left(p\rho_1 + \bar{p}\rho_2\right) d\mu \\
				&= - \int_X p\rho_1 \log \left(p\rho_1 + \bar{p}\rho_2\right) d\mu - \int_X \bar{p}\rho_2 \log \left(p\rho_1 + \bar{p}\rho_2\right) d\mu \\
				&\leq  - \int_X p\rho_1 \log p\rho_1 d\mu - \int_X \bar{p}\rho_2 \log \bar{p}\rho_2 d\mu\\
				&=  - \int_X p\rho_1 \log p d\mu - \int_X p\rho_1 \log \rho_1 d\mu - \int_X \bar{p}\rho_2 \log \bar{p} d\mu - \int_X \bar{p}\rho_2 \log \rho_2 d\mu\\
				&=  - p \log p \int_X \rho_1 d\mu - p \int_X \rho_1 \log \rho_1 d\mu - \bar{p} \log \bar{p} \int_X \rho_2 d\mu - \bar{p} \int_X \rho_2 \log \rho_2 d\mu\\
				&=  - p \log p - \bar{p} \log \bar{p} + p S(\rho_1) + \bar{p} S(\rho_2)\\
			\end{aligned}
		\end{equation}
		The equality holds if and only if $\rho_2=0$ wherever $\rho_1\neq0$ and $\rho_1=0$ wherever $\rho_2\neq0$. That is, the equality holds if and only if the two distributions have disjoint support. Therefore orthogonal distributions are exactly distributions with disjoint support.
		
		Suppose $\rho_4$ has disjoint support from $\rho_1 = p \rho_2 + \bar{p} \rho_3$, then it has disjoint support from $\rho_2$ and $\rho_3$ because the support of $\rho_1$ is the union of the supports of $\rho_2$ and $\rho_3$. Conversely, if $\rho_4$ has disjoint support from both $\rho_2$ and $\rho_3$, then $\rho_4$ has disjoint support from $\rho_1$ as well. Therefore mixtures preserve orthogonality.
		
		All these arguments are valid for discrete classical ensemble spaces, changing integrals to sums.
		
		For the quantum case, we haven't found a short proof that does not require defining the KL divergence and the entropy for a joint distribution. The result, however, is generally known and can be found, for example, in \href{https://www.cambridge.org/highereducation/books/quantum-computation-and-quantum-information/01E10196D0A682A6AEFFEA52D53BE9AE}{Nielsen and Chuang}.
	\end{proof}
\end{mathSection}


\subsection{Uniqueness of entropy of mixing coefficients}

The axiom of entropy imposes only the existence of a universal function $I(p,\bar{p})$ without specifying its functional form. We are now going to show that the functional form is actually already fixed by the axiom: the Shannon entropy $-p \log p - \bar{p} \log \bar{p}$ is in fact the only function that is satisfies the axiom. The result is achieved by calculating the entropy of the same final ensemble as decomposed into a mixture in different ways. Since the final entropy should not care about how the mixture is performed, $I(p,\bar{p})$ must satisfy some properties that lead to the final results.

We note that the Shannon entropy in our framework does not represent an absolute entropy, but the maximal entropy increase during mixing. This is why we are able to keep the framework general. In fact, note that the proof does not technically know whether we are in a classical or quantum ensemble space, or even in an ensemble space of a new possible theory.

\begin{mathSection}
\begin{thrm}[Uniqueness of entropy]\label{pm-es-entropyUnique}
	If there exists an ensemble space with an infinite set of orthogonal ensembles, then the entropy of the coefficients $I(p,\bar{p})$ is the Shannon entropy. That is, $I(p,\bar{p}) = - \kappa \left(p \log p + \bar{p} \log \bar{p}\right)$ where $\kappa>0$ is the arbitrary multiplicative constant for the entropy. For a mixture of arbitrarily many elements, $I(\{p_i\}) = - \kappa \sum_i p_i \log p_i$.
\end{thrm}

\begin{proof}
	Since the upper entropy bound has to be the same for all spaces, let us assume that $\Ens$ is such that it contains countably many orthogonal ensembles $\{\ens_l\}_{l=1}^{\infty}$. Since mixtures preserve orthogonality, for any convex combinations $\sum_i p_i \ens[a]_i$ of finitely many $\{\ens[a]_i\}_{i=1}^{n} \subset \{\ens_l\}_{l=1}^{\infty}$,  we have $S(\sum_i p_i \ens[a]_i ) = I_n(p_1, p_2, \dots, p_n) + \sum_i p_i S(\ens[a]_i)$ where $I_n : [0,1]^n \to \mathbb{R}$ is a function of the coefficients only. Note that, given commutativity, the order of the $p_i$ does not matter and, since the coefficients can be zero, we must have $I_n(p_1, p_2, \dots, p_n) = I_{n+1}(p_1, p_2, \dots, p_n, 0)$. Therefore we can think of $I$ as a function of the coefficients and write $I(\{p_i\}_{i=1}^{n})$. Note that since we can write $I(\{p_i\}_{i=1}^{n}) = S(\sum_i p_i \ens[a]_i ) - \sum_i p_i S(\ens[a]_i)$, and both entropy and mixing are continuous, then $I$ is continuous.
	
	We now show that $I\left(\left\{\frac{1}{n}\right\}_{i=1}^{n}\right) = \kappa \log n$ with $\kappa > 0$. That is, the maximum increase of entropy for a uniform distribution is proportional to the logarithm of the number of cases. Pick two positive integers $n, m \in \mathbb{Z}^+$. Pick $nm$ elements $\ens[a]_{jk} \in \{e_l\}_{l=1}^{\infty}$ where $1\leq j \leq n$ and $1 \leq  k \leq m$. We have:
	\begin{equation}
		\begin{aligned}
			S\left(\sum_{j=1}^{n}\sum_{k=1}^{m} \frac{1}{n}\frac{1}{m} \ens[a]_{jk}\right) &= I\left(\left\{ \frac{1}{n}\frac{1}{m}\right\}_{i=1}^{nm}\right) + \sum_{j=1}^{n}\sum_{k=1}^{m} \frac{1}{n}\frac{1}{m} S\left(\ens[a]_{jk}\right) \\
			= S\left(\sum_{j=1}^{n} \frac{1}{n} \sum_{k=1}^{m} \frac{1}{m} \ens[a]_{jk}\right)  &=I\left(\left\{ \frac{1}{n}\right\}_{i=1}^{n}\right) + \sum_{j=1}^{n} \frac{1}{n} S\left(\sum_{k=1}^{m}\frac{1}{m}\ens[a]_{jk}\right) \\
			&=I\left(\left\{ \frac{1}{n}\right\}_{i=1}^{n}\right) + \sum_{j=1}^{n} \frac{1}{n}\left(I\left(\left\{ \frac{1}{m}\right\}_{i=1}^{m}\right) +  \sum_{k=1}^{m}\frac{1}{m}S\left(\ens[a]_{jk}\right)\right) \\
			&=I\left(\left\{ \frac{1}{n}\right\}_{i=1}^{n}\right) + I\left(\left\{ \frac{1}{m}\right\}_{i=1}^{m}\right) + \sum_{j=1}^{n} \sum_{k=1}^{m}\frac{1}{n}\frac{1}{m}S\left(\ens[a]_{jk}\right).
		\end{aligned}
	\end{equation}
	Therefore
	\begin{equation}
		\begin{aligned}
			I\left(\left\{ \frac{1}{nm}\right\}_{i=1}^{nm}\right) &=I\left(\left\{ \frac{1}{n}\right\}_{i=1}^{n}\right) + I\left(\left\{ \frac{1}{m}\right\}_{i=1}^{m}\right).
		\end{aligned}
	\end{equation}
	Note that $f(n) = I\left(\left\{ \frac{1}{n}\right\}_{i=1}^{n}\right)$ is a function of $n$ only, such that $f(nm) = f(n) + f(m)$. Since $I$ is continuous, by \href{https://en.wikipedia.org/wiki/Cauchy%27s_functional_equation}{Cauchy's functional equation} we have $f(n) = \kappa \log n$. Since the entropy is strictly concave, $\kappa$ must be positive. Therefore
	\begin{equation}
		I\left(\left\{ \frac{1}{n}\right\}_{i=1}^{n}\right) = \kappa \log n
	\end{equation}
	for some $\kappa>0$.
	
	We now show that if the coefficients $p_i$ are rationals, $I_n\left(\left\{ p_i\right\}_{i=1}^{n}\right) = - \kappa \sum_{i=1}^{n} p_i \log p_i$. Let $\{p_i\}_{i=1}^{n}$ be rational coefficients for a convex combination. We can write them as $p_i = \frac{m_i}{m}$ where $\{m_i\}, m \in \mathbb{Z}^+$ and $m$ is the least common denominator. Since $p_i$ are the coefficients of a convex combination, we must have $\sum_{i=1}^{n} m_i = m$. Since $m_i$ is a positive integer, we can write $m_i = \sum_{j=1}^{m_i} 1$. We now take $m$ orthogonal ensembles $\ens[a]_{ij}$ where $1 \leq i \leq n$ and $1 \leq j \leq m_i$. We have
	\begin{equation}
		\begin{aligned}
			S\left(\sum_{i=1}^{n}\sum_{j=1}^{m_i} \frac{1}{m} \ens[a]_{ij}\right) &= I\left(\left\{ \frac{1}{m}\right\}_{i=1}^{m}\right) + \sum_{i=1}^{n}\sum_{j=1}^{m_i} \frac{1}{m} S\left(\ens[a]_{ij}\right) \\
			&= \kappa \log m + \sum_{i=1}^{n}\sum_{j=1}^{m_i} \frac{1}{m} S\left(\ens[a]_{ij}\right) \\
		\end{aligned}
	\end{equation}
	\begin{equation}
		\begin{aligned}
			= S\left(\sum_{i=1}^{n} \frac{m_i}{m} \sum_{j=1}^{m_i} \frac{1}{m_i} \ens[a]_{ij}\right)  &=I\left(\left\{ \frac{m_i}{m}\right\}_{i=1}^{n}\right) + \sum_{i=1}^{n} \frac{m_i}{m}  S\left(\sum_{j=1}^{m_i} \frac{1}{m_i} \ens[a]_{ij}\right) \\
			&=I\left(\left\{ p_i\right\}_{i=1}^{n}\right) + \sum_{i=1}^{n} \frac{m_i}{m}\left(I\left(\left\{ \frac{1}{m_i}\right\}_{i=1}^{m_i}\right) + \sum_{j=1}^{m_i} \frac{1}{m_i} S\left(\ens[a]_{ij}\right)\right) \\
			&=I\left(\left\{ p_i\right\}_{i=1}^{n}\right) + \sum_{i=1}^{n} \frac{m_i}{m} I\left(\left\{ \frac{1}{m_i}\right\}_{i=1}^{m_i}\right) + \sum_{i=1}^{n} \frac{m_i}{m} \sum_{j=1}^{m_i} \frac{1}{m_i}S\left(\ens[a]_{ij}\right) \\
			&=I\left(\left\{ p_i\right\}_{i=1}^{n}\right) + \sum_{i=1}^{n} p_i \kappa \log m_i + \sum_{i=1}^{n}  \sum_{j=1}^{m_i} \frac{1}{m} S\left(\ens[a]_{ij}\right).
		\end{aligned}
	\end{equation}
	Therefore
	\begin{equation}
		\begin{aligned}
			\kappa \log m &= I\left(\left\{ p_i\right\}_{i=1}^{n}\right) + \sum_{i=1}^{n} p_i \kappa \log m_i \\
			I\left(\left\{ p_i\right\}_{i=1}^{n}\right) &= \kappa \log m - \sum_{i=1}^{n} p_i \kappa \log m_i = \sum_{i=1}^{n} p_i \kappa \log m - \sum_{i=1}^{n} p_i \kappa \log m_i \\
			&= - \sum_{i=1}^{n} p_i \kappa \log \frac{m_i}{m} = - \kappa \sum_{i=1}^{n} p_i \log p_i.
		\end{aligned}
	\end{equation}
	
	Lastly, let $\{p_i\}_{i=1}^{n}$ be coefficients for a convex combination, not necessarily rational. Since $I$ is continuous and $p_i$ can be approximated with rational values to an arbitrary level of precision, we will have $I\left(\left\{ p_i\right\}_{i=1}^{n}\right) = - \kappa \sum_{i=1}^{n} p_i \log p_i$. In the case of $n=2$, we have $I(p_1, p_2) = - \kappa p_1 \log p_1 - \kappa p_2 \log p_2$.	
\end{proof}

\begin{coro}
	The unit for the entropy is determined (up to the physical dimension) by the maximum of the entropy of the coefficients $I\left(\frac{1}{2}, \frac{1}{2}\right)$. If the entropy is measured in bits, then $I\left(\frac{1}{2}, \frac{1}{2}\right) = 1$.
\end{coro}

\begin{proof}
	A rescaling of the entropy will also rescale the entropy of the coefficients. Therefore setting the value of the maximum of $I$ will set the arbitrary multiplicative factor. If $I\left(\frac{1}{2}, \frac{1}{2}\right) = 1$, then $\kappa$ is equal to one and the logarithm is base two, which corresponds to the entropy measured in bits. Note that this does not fix the physical dimensions of the entropy, only the numerical value.
\end{proof}
\end{mathSection}

\subsection{Separateness and orthogonality}

Recall that orthogonality is formally defined in terms of the entropy: two ensembles are orthogonal if the entropy is maximally increased during mixture. The name was chosen because it will recover the usual notion of orthogonality for the space of distributions in both classical and quantum mechanics. Like the standard notion of orthogonality, in fact, it is irreflexive (no ensemble is orthogonal to itself) and symmetric (if $\ens[a]$ is orthogonal to $\ens[b]$, then $\ens[b]$ is orthogonal to $\ens[a]$). Moreover, an ensemble is not orthogonal to its own components.

% TODO for JSD: Apart from the mathematical analogy, we can see how the notion of an inner product is conceptually linked to the notion of ensemble. Consider two ensembles $\ens[a]$ and $\ens[b]$ as the two infinite collection of outputs of two preparation procedures. These are orthogonal if the ensembles are mutually exclusive: they have no instance in common. If Amanda selects one of the two preparation procedures, it may take Boris only one instance to be able to determine which preparation was selected. A full bit of information can be transferred. Now suppose $\ens[a]$ and $\ens[b]$ are distinct, but not mutually exclusive. Then the two ensembles will have some instances in common. If the instance produced by the preparation Amanda selects is one in common, then Boris cannot determine the preparation with only one measurement. The more the two ensembles are similar to each other, the less the entropy will increase during mixing, and it will not increase

Recovering the math, however, would be meaningless if we didn't also provide a stronger conceptual model. As we said, two ensembles are orthogonal if they are mutually exclusive: they have no instance in common. This means that if Amanda selects between two preparation procedures that correspond to orthogonal ensembles, Boris may only need one instance to determine which preparation was selected. A full bit of information can be transferred. This is why optimal measurements are defined across mutually exclusive, i.e. orthogonal, outcomes.

Moreover, recall that two ensembles are separate if they do not have a common component, they are not different mixtures of the same ensemble. Clearly, if two ensembles do not have instances in common, they cannot be the mixture of the same ensemble. In fact, from the axioms we posed, we are able to recover that orthogonality implies separateness. One may be tempted to conclude the converse: if two ensembles are not orthogonal, they have some instances in common and therefore we can group those common instances into a sub-ensemble of both. However, and this is the crucial problem, there is nothing that guarantees us that we can find a preparation procedure that spans only the overlap. In classical mechanics, the assumption is that we can always do it. In quantum mechanics, this does not always work and, with these concepts, it is easy to see why.

Quantum states have a lower bound on the entropy. If two pure states have an overlap, the ensemble that would corresponding to that overlap would have variability lower than a pure state: it would correspond to an entropy lower than that of a pure state. That is, we cannot produce the overlap with a reliable preparation procedure. The ensembles have instances in common but do not have an ensemble in common. This means that we have ensembles that are not mutually exclusive (i.e. not orthogonal) but do not have a common component (i.e. are separate). Understanding the difference between classical and quantum mechanics lies in understanding this case.

\begin{mathSection}
\begin{prop}\label{pm-es-orthoProps}
	Orthogonality satisfies the following properties:
	\begin{enumerate}
		\item irreflexivity: $\ens[a] \northo \ens[a]$
		\item symmetry: $\ens[a] \ortho \ens[b]$ if and only if $\ens[b] \ortho \ens[a]$
		\item components are not orthogonal: if $\ens[b]$ is a component of $\ens[a]$ then $\ens[a] \northo \ens[b]$
		\item orthogonality implies separateness: if $\ens[a] \ortho \ens[b]$ then $\ens[a] \separate \ens[b]$.
	\end{enumerate}
\end{prop}

\begin{proof}
	For 1, let $\ens[a] \in \Ens$. We have $S(p\ens[a] + \bar{p}\ens[a]) = S(\ens[a]) < I(p,\bar{p}) + p S(\ens[a]) + \bar{p} S(\ens[a])$. Therefore $\ens[a]$ is not orthogonal to itself as it does not saturate the upper bound.
	
	For 2, note that the upper entropy bound is symmetric in $\ens[a]$ and $\ens[b]$.
	
	For 3, let $\ens[a] = p \ens[b] + \bar{p} \ens[c]$. Since mixtures preserve orthogonality, $\ens[b] \ortho \ens[a]$ if and only if $\ens[b] \ortho \ens[b]$ and $\ens[b] \ortho \ens[c]$. But $\ens[b]$ is not orthogonal to itself, therefore $\ens[a]$ and $\ens[b]$ are not orthogonal.
	
	For 4, we demonstrate the contrapositive: that ensembles that are not separate are not orthogonal. Let $\ens[a], \ens[b] \in \Ens$ have a common component. That is, $\ens[a] = p \ens[c] + \bar{p} \ens[d]$ and $\ens[b] = \lambda \ens[c] + \bar{\lambda} \ens[e]$. Since mixtures preserve orthogonality, $\ens[a] \ortho \ens[b]$ if and only if $\ens[a] \ortho \ens[c]$ and $\ens[a] \ortho \ens[e]$. But $\ens[c]$ is a component of $\ens[a]$, therefore they are not orthogonal. Therefore two ensembles that have a common component are not orthogonal. This means that if two ensembles are orthogonal they cannot have a common component and are therefore separate.
\end{proof}
\end{mathSection}

As we did for separateness, we extend the notion of orthogonality to sets of ensembles.

\begin{mathSection}
\begin{defn}
	Two sets of ensembles are orthogonal if all the elements of one are orthogonal to all the elements of the other. That is, $A \ortho B$ with $A, B \subseteq \Ens$ if $\ens[a] \ortho \ens[b]$ for all $\ens[a] \in A$ and $\ens[b] \in B$. 
\end{defn}

\begin{prop}
	Let $A, B \subseteq \Ens$ be two sets of ensembles such that $A \ortho B$. Then the following are true:
	\begin{enumerate}
		\item the two sets are separate: $A \separate B$
		\item their hulls are orthogonal: $\hull(A) \ortho \hull(B)$, $\shull(A) \ortho \shull(B)$ and  \\ $\chull(A) \ortho \chull(B)$
	\end{enumerate}
\end{prop}

\begin{proof}
	For 1, by definition $\ens[a] \ortho \ens[b]$ for all $\ens[a] \in A$ and $\ens[b] \in B$. Since orthogonality implies separateness, we also have $\ens[a] \separate \ens[b]$.
	
	For 2, we will concentrate on the closed hull, since if two sets are orthogonal so are their subsets. Since mixtures preserve orthogonality, every finite mixture of $A$ is orthogonal to every finite mixture of $B$. To show that this extends to the topological closure, let $\ens[a]_i \in A$ and $\ens[b]_j \in B$ be two sequences of finite mixtures that converge in $A$ and $B$ respectively. Consider $f(p,\ens[a]_i, \ens[b]_j) = S(p \ens[a]_i + \bar{p} \ens[b]_j) - p S(\ens[a]_i) - \bar{p} S(\ens[b]_j)$. Since all mixtures are orthogonal, $f(p,\ens[a]_i, \ens[b]_j) = I(p,\bar{p})$. Note that $f$ is a continuous function, therefore the limits will also converge to $I(p,\bar{p})$. This means that every element in the closed hull of $A$ is orthogonal to every element in the closed hull of $B$.
\end{proof}
\end{mathSection}

We may want to extend the notion of separate decomposability to orthogonal decomposabilty, though it is not clear whether this will be useful or not. 

\begin{mathSection}
\begin{defn}
	An ensemble is \textbf{orthogonally decomposable} if it can be expressed as a mixture of two orthogonal ensembles. An ensemble is \textbf{orthogonally monodecomposable} if it is both orthogonally decomposable and monodecomposable.
\end{defn}

\begin{coro}
	An ensemble that is orthogonally decomposable is also separately decomposable.
\end{coro}

\begin{proof}
	Since orthogonality implies separateness by \ref{pm-es-orthoProps}, an orthogonal decomposition is also a separate decomposition.
\end{proof}
\end{mathSection}

\section{Affine combinations and vector space embedding}

In this section we will see how the bounds of entropy constrain ensemble spaces to embed into vector spaces. Moreover, the ensemble space must be bounded along any direction. In other words, the existence of an entropy rules out physically pathological cases that a convex space may otherwise allow.

%First of all, the bounds make it possible to define a unique inverse for statistical mixture. This allows us to define affine combinations, and to show that the ensemble space is embedded into a vector space. Lastly, the bounds will force this embedding to be into a subset of a bounded set of the vector space.

\subsection{Affine combinations}

The axiom of mixture just tells us that given two ensembles and a mixing coefficient, we can identify the mixture of the two ensembles. It does not guarantee the reverse. That is, given a final mixed ensemble, the mixing ratio and one component, it is not necessarily true that the second component is uniquely determined. Mathematically, the uniqueness of this type of decomposition is called the cancellative property of a convex space. This property is important as it can be shown that it is necessary and sufficient for the convex space to embed into a vector space (see \href{https://arxiv.org/abs/1105.1270}{arXiv:1105.1270}).

In an ensemble space, the continuity and the strict concavity of the entropy forces the space to be cancellative. If $p \ens[a] + \bar{p} \ens = p \ens[b] + \bar{p} \ens$ for some $p \in (0,1)$, from the convex structure we can show that $p(\lambda\ens[a] + \bar{\lambda}\ens[b]) + \bar{p} \ens = p\ens[a] + \bar{p} \ens = p \ens[b] + \bar{p} \ens$ for all $p \in (0,1)$ and $\lambda \in [0,1]$. That is, if mixing $\ens$ with either $\ens[a]$ or $\ens[b]$ for some mixing coefficient $p$ yields the same result, then all non-trivial mixtures of $\ens$ with any mixture of $\ens[a]$ and $\ens[b]$ will yield the same result. But this means that the entropy cannot change as we change $\ens[a]$ to $\ens[b]$ during mixture, which can only happen if they are the same ensemble.

\begin{mathSection}
\begin{defn}
	A convex space $X$ is \textbf{cancellative} if $p \ens[a] + \bar{p} \ens = p \ens[b] + \bar{p} \ens$ for some $p \in (0,1)$ implies $\ens[a] = \ens[b]$.
\end{defn}

\begin{thrm}[Ensemble spaces are cancellative]
	Let $\Ens$ be an ensemble space. Let $\ens[a], \ens[b], \ens \in \Ens$ such that $p\ens[a] + \bar{p} \ens = p \ens[b] + \bar{p} \ens$ for some $p \in (0,1)$. Then $\ens[a] = \ens[b]$.
\end{thrm}

\begin{proof}
	Let $\ens[a], \ens[b], \ens \in \Ens$ such that $p_0\ens[a] + \bar{p}_0 \ens = p_0 \ens[b] + \bar{p}_0 \ens$ for some $p_0 \in (0,1)$.
	
	First, we show that $p\ens[a] + \bar{p} \ens = p \ens[b] + \bar{p} \ens$ for all $p \in (0,p_0]$. In that case, since $0 < \frac{p}{p_0} \leq 1$, we have
	\begin{equation}
		\begin{aligned}
			p \ens[a] + \bar{p} \ens &= \frac{p}{p_0} (p_0 \ens[a] + \bar{p}_0 \ens) + \overline{\left(\frac{p}{p_0}\right)} \ens = \frac{p}{p_0} (p_0 \ens[b] + \bar{p}_0 \ens) + \overline{\left(\frac{p}{p_0}\right)} \ens = p \ens[b] + \bar{p} \ens.
		\end{aligned}
	\end{equation}
	
	Now we show that $p\ens[a] + \bar{p} \ens = p \ens[b] + \bar{p} \ens$ for all $p \in (0,1)$. Since we want to be able to expand multiple times, we want to be able to find a $p \in (0,1)$ such that
	\begin{equation}
		p \ens[a] + p \ens[a] + (1-2p) \ens = p \ens[a] + \bar{p} (p_0 \ens[a] + \bar{p}_0 \ens)= p \ens[a] + \bar{p} p_0 \ens[a] + \bar{p} \bar{p}_0 \ens.
	\end{equation}
	The coefficient of the middle term on both sides of the equality, then, has to match. That is, we want $p = \bar{p} p_0$, which means
	\begin{equation}
		\begin{aligned}
			p&=(1-p)p_0=p_0 - p p_0 \\
			p_0 &= p + p p_0 = (1+p_0) p  \\
			p &= \frac{p_0}{1+p_0}
		\end{aligned}
	\end{equation}
	Note that since $p_0 > 0$ we have $p>0$, and since the numerator is always less than the denominator $p < 1$. We have
	\begin{equation}
		\begin{aligned}
			2p \ens[a] + \overline{2p} \ens &=  p \ens[a] + p \ens[a] + (1-2p) \ens =  p \ens[a] + \bar{p} (p_0 \ens[a] + \bar{p}_0 \ens) = p \ens[a] + \bar{p} (p_0 \ens[b] + \bar{p}_0 \ens) \\
			&= p \ens[a] + p \ens[b] + (1-2p) \ens = p \ens[b] + p \ens[a] + (1-2p) \ens = p \ens[b] + \bar{p} (p_0 \ens[a] + \bar{p}_0 \ens) \\
			&= p \ens[b] + \bar{p} (p_0 \ens[b] + \bar{p}_0 \ens) = p \ens[b] + p \ens[b] + (1-2p) \ens \\
			&= 2p \ens[b] + \overline{2p} \ens
		\end{aligned}
	\end{equation}
	The relationship, then, is valid for $p_1 = 2p$. Note that $p_1 > p_0$. In fact
	\begin{equation}
		\begin{aligned}
			p_1 &= \frac{2p_0}{1+p_0} > p_0  \\
			2 p_0 &> (1+p_0) p_0 = p_0 + p_0^2 \\
			p_0 &> p_0^2 \\
		\end{aligned}
	\end{equation}
	which is true since $p_0 \in (0,1)$. Since now the relationship holds for $p_1 = \frac{2 p_0}{1+p_0} > p_0 $, we can repeat the process again and find that it holds for $p_2 = \frac{2 p_1}{1+p_1} > p_1$ and so on. We thus have a sequence of elements between $0$ and $1$ and we need to determine the limit of this sequence. The only two fixed points of the expression $f(x) = \frac{2x}{1+x}$ are $0$ and $1$, with $0$ being a repelling fixed point and $1$ an attracting fixed point. Since we start with an element that is strictly between those values, the sequence will converge to $1$. Therefore, since we can always find a greater mixing coefficient for which the relationship holds, combined with the previous result, $p\ens[a] + \bar{p} \ens = p \ens[b] + \bar{p} \ens$ for all $p \in (0,1)$.
	
	Now we show that if  $p\ens[a] + \bar{p} \ens = p \ens[b] + \bar{p} \ens$ for all $p \in (0,1)$, then we also have $p(\lambda\ens[a] + \bar{\lambda}\ens[b]) + \bar{p} \ens = p\ens[a] + \bar{p} \ens = p \ens[b] + \bar{p} \ens$ for all $\lambda \in [0,1]$. We have
	\begin{equation}
		\begin{aligned}
			p(\lambda\ens[a] + \bar{\lambda}\ens[b]) + \bar{p} \ens &= \lambda(p \ens[a] + \bar{p} \ens) + \bar{\lambda}(p\ens[b] + \bar{p} \ens) = \lambda(p \ens[a] + \bar{p} \ens) + \bar{\lambda}(p\ens[a] + \bar{p} \ens) \\
			&= p\ens[a] + \bar{p} \ens = p\ens[b] + \bar{p} \ens 
		\end{aligned}
	\end{equation}
	
	Lastly, we show that $S(\lambda \ens[a] + \bar{\lambda} \ens[b]) = S(\ens[a]) = S(\ens[b])$ for all $\lambda \in [0,1]$. Using the entropy bounds, with $\lambda \in [0,1]$, we have
	\begin{equation}
		\begin{aligned}
			\lim\limits_{p \to 1} S(p(\lambda\ens[a] + \bar{\lambda}\ens[b]) + \bar{p}\ens) &\leq \lim\limits_{p \to 1} \left[ I(p,\bar{p}) + p S(\lambda\ens[a] + \bar{\lambda}\ens[b]) + \bar{p} S(\ens) \right] = S(\lambda\ens[a] + \bar{\lambda}\ens[b]) \\
			\lim\limits_{p \to 1} S(p(\lambda\ens[a] + \bar{\lambda}\ens[b]) + \bar{p}\ens) &\geq \lim\limits_{p \to 1} \left[ p S(\lambda\ens[a] + \bar{\lambda}\ens[b]) + \bar{p} S(\ens) \right] = S(\lambda\ens[a] + \bar{\lambda}\ens[b])
		\end{aligned}
	\end{equation}
	and therefore
	\begin{equation}
		\begin{aligned}		
			\lim\limits_{p \to 1} S(p(\lambda\ens[a] + \bar{\lambda}\ens[b]) + \bar{p}\ens) &= S(\lambda\ens[a] + \bar{\lambda}\ens[b]).
		\end{aligned}
	\end{equation}
	Using the above property, we also have
	\begin{equation}
		\begin{aligned}
			S(\lambda\ens[a] + \bar{\lambda}\ens[b]) &= \lim\limits_{p \to 1} S(p(\lambda\ens[a] + \bar{\lambda}\ens[b]) + \bar{p}\ens) = \lim\limits_{p \to 1} S(p \ens[a] +\bar{p} \ens) = S(\ens[a]) \\
			&= \lim\limits_{p \to 1} S(p \ens[b] +\bar{p} \ens) = S(\ens[b]) \\
		\end{aligned}
	\end{equation}
	Since $S(\lambda\ens[a] + \bar{\lambda}\ens[b]) = \lambda S(\lambda\ens[a] + \bar{\lambda}\ens[b]) + \bar{\lambda} S(\lambda\ens[a] + \bar{\lambda}\ens[b]) = \lambda S(\ens[a]) + \bar{\lambda} S(\ens[b])$, $\ens[a] = \ens[b]$ by strict concavity.
\end{proof}
\end{mathSection}

The fact that the convex space is cancellative essentially allows us to ``invert'' a convex combination, allowing affine combinations. An affine combination is a linear combination where the coefficients sum to one, like in a convex combination, but coefficients can be negative, unlike in a convex combination. Conceptually we can ``take part of an ensemble out'' from another ensemble. The ability in quantum mechanics to use negative pseudo-probability, for example in Wigner functions, stems from the fact that the space of ensembles allows affine combinations because it is cancellative.

An affine combination can be translated into the equality between two convex combinations by moving all the negative coefficients on the other side of the equality. This reduces to finding the component a mixture, which, if it exists, is unique by the cancellative property.

It is important to note that while we can write affine combinations, not all affine combinations correspond to ensembles. For example, if $\ens[b]$ is not a component of $\ens[a]$, then $\frac{3}{2}\ens[a] - \frac{1}{2} \ens[b]$ cannot possibly be an ensemble as we cannot take any ``amount of $\ens[b]$'' from $\ens[a]$.

\begin{mathSection}	
\begin{defn}[Affine combinations]
	Let $\{\ens_i\}_{i=1}^n \subseteq \Ens$ be a finite sequence of ensembles and $\{r_i\}_{i=1}^n \subseteq \mathbb{R}$ be a finite sequence of coefficients such that $\sum_{i=1}^n r_i = 1$. The \textbf{affine combination} $\sum_{i=1}^n r_i \ens_i$ is the ensemble $\ens[a] \in \Ens$, if it exists, such that  $\sum_{i \in I} \frac{r_i}{r} \ens_i = \frac{1}{r} \ens[a] + \sum_{i \notin I} \frac{-r_i}{r} \ens_i$ where $I = \{ i \in [1,n] \, | \, r_i \geq 0 \}$ and $r = \sum_{i \in I} r_i$.
\end{defn}

\begin{check}
	For the definition to work, we need to show that the affine combination can be re-expressed in terms of convex combinations. Note that the coefficients $r_i$ are not necessarily positive, but still sum to $1$. With the given definitions, $I$ includes exactly the indexes that correspond to non-negative coefficients, and $r$ is the sum of all those non-negative coefficients. This means that $\frac{r_i}{r} \in [0,1]$ for all $i \in I$ and $\sum_{i \in I} \frac{r_i}{r}=1$. Therefore $\sum_{i \in I} \frac{r_i}{r} \ens_i$ is a convex combination and it is always well-defined. Additionally, we have that $\sum_{i \in I} r_i = 1 - \sum_{i \notin I} r_i$, which means $\frac{-r_i}{r - 1} \in [0,1]$ for all $i \notin I$ and $\sum_{i \in I} \frac{-r_i}{r-1}=1$. Therefore $\sum_{i \in I} \frac{-r_i}{r-1} \ens_i$ is a convex combination and it is always well-defined. Note that $\ens[a]$ is defined to be such that $\sum_{i \in I} \frac{r_i}{r} \ens_i = \frac{1}{r} \ens[a] + \frac{r-1}{r}\sum_{i \notin I} \frac{-r_i}{r-1} \ens_i$. If it exists, it is unique.
\end{check}	

\begin{remark}
	In the same way that not all infinite convex combinations yield valid ensembles, not all finite or infinite affine combinations yield valid ensembles.
\end{remark}
\end{mathSection}

We still need to understand how to extend affine combination to the infinite case. The safest way is to require the two infinite mixtures $\sum_{i \in I} \frac{r_i}{r} \ens_i$ and $\sum_{i \notin I} \frac{-r_i}{r} \ens_i$ to converge and then require that the resulting affine problem admit a solution.

\subsection{Vector space embedding}

As we said before, the cancellative property is necessary and sufficient for the embedding of a convex space into a vector space. While this can be proved with a short mathematical proof, it would be an abstract construction which would not give us any insight into what is actually being represented physically. Therefore, after choosing an internal point $\ens[a] \in \Ens$ as an origin, we will construct a vector space from the ground up, as the space of differences between ensembles. This will also justify why, locally at every point, this space becomes the space of variations.

\begin{figure}[H]
	\centering
	\begin{tikzpicture}
		%\draw [help lines] (0,0) grid (4,4);
		
		\coordinate (label1) at (6.7,3);
		\coordinate (label2) at (6.7,2);
		\coordinate (label3) at (6.7,1);
		\coordinate (a) at (2,2);
		\coordinate (b) at (4,3);
		\coordinate (c) at ($(a)!-1!(b)$);
		\coordinate (d) at ($(a)!0.666!(b)$);
		\coordinate (e) at (4.5,1.5);
		\coordinate (f) at ($(b)!0.5!(e)$);
		\coordinate (g) at ($(a)!2!(f)$);
		\draw[-Stealth] (a) -- (b);
		\draw[-Stealth] (a) -- (c);
		\draw[-Stealth] (a) -- (d);
		\draw[-Stealth] (a) -- (e);
		\draw[-Stealth] (a) -- (f);
		\draw[dashed] (b) -- (e);
		\draw[dashed] (b) -- (g) -- (e);
		\draw[dashed] (f) -- (g);
		\filldraw [black] (a) circle (1pt) node[below] {$\ens[a]$};
		\filldraw [black] (b) circle (1pt) node[above right] {$\ens[b]$};
		\filldraw [black] (c) circle (1pt) node[left] {$\ens[c]$};
		\filldraw [black] (d) circle (1pt) node[above] {$\ens[d]$};
		\filldraw [black] (e) circle (1pt) node[below right] {$\ens[e]$};
		\filldraw [black] (f) circle (1pt) node[above right] {$\ens[f]$};
		
		\node at (label1) [right] {$\ens[a] = \frac{1}{2} \ens[b] + \frac{1}{2} \ens[c] \iff 1(\ens[c]-\ens[a]) = -1(\ens[b]-\ens[a])$};
		
		\node at (label2) [right] {$\ens[d] = \frac{1}{3} \ens[a] + \frac{2}{3} \ens[b] \iff 1(\ens[d]-\ens[a]) = \frac{2}{3}(\ens[b]-\ens[a])$};
		
		\node at (label3) [right] {$\ens[f] = \frac{1}{2} \ens[b] + \frac{1}{2} \ens[e] \iff 1(\ens[f]-\ens[a]) = \frac{1}{2}(\ens[b]-\ens[a]) + \frac{1}{2}(\ens[e]-\ens[a]) $};
		
	\end{tikzpicture}
	%\includegraphics[width=0.7\textwidth]{tempimages/EnsembleDifferences.jpg}
	\caption{Once a reference point is picked (i.e. $\ens[a]$ in the figure), the differences with respect to that point behave like elements of a vector space. The relationship between ensembles in terms of mixtures become vector space relationships between differences. Note that the sum is effectively a ``half parallelogram law,'' so that the result always is a member of the convex space.}\label{pm-es-vectorJustification}
\end{figure}

Figure \ref{pm-es-vectorJustification} shows the motivation of the construction. We take an internal point $\ens[a] \in \Ens$ to be a reference, the origin. For any other ensemble $\ens[b]$, we imagine the change from $\ens[a]$ to $\ens[b]$ as the arrow that connects the first to the second. This is the ensemble difference $(\ens[b] - \ens[a])$. The negative of that difference would find an ensemble $\ens[c]$ in the opposite direction. Note that $\ens[a]$ now sits exactly in between $\ens[b]$ and $\ens[c]$, and therefore $\ens[a] = \frac{1}{2}\ens[b] + \frac{1}{2}\ens[c]$ if and only if $(\ens[b] - \ens[a]) = - (\ens[c]- \ens[a])$. That is, relationships between vectors are re-expressed as convex combinations of ensembles.

Similarly, we can now stretch or shrink differences by a factor. For example, multiplying the difference by a third will give us the ensemble that is at a third between $\ens[a]$ and $\ens[b]$. Similarly, we can add differences. For example, $\frac{1}{2}(\ens[b] - \ens[a]) + \frac{1}{2}(\ens[c] - \ens[a])$ will give us the difference $\left(\left(\frac{1}{2}\ens[b] + \frac{1}{2}\ens[c]\right) - \ens[a]\right)$, the difference between $\ens[a]$ and the midpoint between $\ens[b]$ and $\ens[c]$. Note that if we double that difference, we obtain the parallelogram law for vector addition.

Clearly, if we stretch a difference too much, we will go out of the ensemble space. Yet, since we can always shrink and then stretch,  all operations defined on differences within the ensemble space can be formally defined on differences that go outside the ensemble space. Given that $\ens[a]$ is an internal point, then differences exist in every direction. This is, in a nutshell, the vector space we are constructing. This clarifies that, in the end, we are always going to be interested in sets of vectors that can be shrunk within the ensemble space.\footnote{It is unclear whether this insight may help us understand what the topology of the embedding vector space is.}

While the motivation is straightforward, the construction is complicated by the fact that there are multiple ways of expressing the same difference. In the third example in figure \ref{pm-es-vectorJustification}, $(\ens[b] - \ens[a])$ and $3(\ens[c] - \ens[a])$ correspond to the same vector. Therefore, we need to specify this equivalence class and prove that the set of equivalence classes forms a vector space. The way that the equivalence relationship is specified is so that the proofs remain short and manageable, and it is therefore mainly a technical issue.\footnote{Originally, the equivalence relationship was specified by three separate cases. While more directly justifiable, that gave 6 different possible combinations to check in every proof.}

\begin{mathSection}
	
	\begin{defn}[Ensemble differences]
		Given an ensemble space, a difference between two ensembles represents the change required to transform one ensemble into another. Formally, an \textbf{ensemble difference}, noted $r(\ens[b] - \ens[a])$, is a triple formed by a real number $r \in \mathbb{R}$ and an ordered pair of ensembles $\ens[a],\ens[b] \in \Ens$.
	\end{defn}
	
	\begin{defn}
		The \textbf{scalar multiplication} of a difference $r(\ens[b] - \ens[a])$ by a real number $s \in \mathbb{R}$, noted $s(r(\ens[b] - \ens[a]))$, is the difference $(sr)(\ens[b] - \ens[a])$
	\end{defn}
	
	\begin{defn}
		Two ensemble differences are \textbf{equivalent}, noted $r(\ens[b] - \ens[a]) \sim s(\ens[c] - \ens[a])$, if there exists $k \in \mathbb{R}$ such that
		$$ \frac{1}{r+s+k} (r \ens[b] + s \ens[a] + k \ens[a]) = \frac{1}{r+s+k} (s \ens[c] + r \ens[a] + k \ens[a]).$$
	\end{defn}
	
	\begin{remark}
		Note that this is an equation between affine combinations since the coefficients can be negative. The equality can be satisfied only if both affine combinations exist.
	\end{remark}
	
	\begin{coro}
		If the above condition is satisfied for some $k$, then it is satisfied for all $k \neq -(r+s)$ for which the affine combination exists.
	\end{coro}
	
	\begin{proof}
		Suppose $\frac{1}{r+s+k} (r \ens[b] + s \ens[a] + k \ens[a]) = \frac{1}{r+s+k} (s \ens[c] + r \ens[a] + k \ens[a])$ for some $k$. Let $k' \in \mathbb{R}$ be such that $\frac{1}{r+s+k'} (r \ens[b] + s \ens[a] + k' \ens[a])$ is a valid affine combination, which requires $r+s+k'\neq0$. Then we have $\frac{1}{r+s+k'} (r \ens[b] + s \ens[a] + k' \ens[a]) = \frac{1}{r+s+k'} (r \ens[b] + s \ens[a] + k \ens[a]) + \frac{k'-k}{r+s+k'} \ens[a] = \frac{r+s+k}{r+s+k'} \frac{1}{r+s+k}(r \ens[b] + s \ens[a] + k \ens[a]) + \frac{k'-k}{r+s+k'} \ens[a] = \frac{r+s+k}{r+s+k'} \frac{1}{r+s+k}(s \ens[c] + r \ens[a] + k \ens[a]) + \frac{k'-k}{r+s+k'} \ens[a] = \frac{1}{r+s+k'} (s \ens[c] + r \ens[a] + k' \ens[a])$.
	\end{proof}
	
	\begin{prop}
		Difference equivalence is reflexive, symmetric and transitive and therefore is an equivalence relation.
	\end{prop}
	
	\begin{proof}
		For reflexivity, we have $\frac{1}{r+r+k} (r \ens[b] + r \ens[a] + k \ens[a]) = \frac{1}{r+r+k} (r \ens[b] + r \ens[a] + k \ens[a])$, which is satisfied for any $\ens[a], \ens[b] \in \Ens$ and $r \in \mathbb{R}$.
		
		For symmetry, note that if $r$ is switched with $s$ and $\ens[b]$ with $\ens[c]$, the left side becomes the right side and vice-versa.
		
		For transitivity, suppose $r(\ens[b] - \ens[a]) \sim s(\ens[c] - \ens[a])$ and $s(\ens[c] - \ens[a]) \sim t(\ens[d] - \ens[a])$. Then for some $k$ and $l$ we have $\frac{1}{r+s+k} (r \ens[b] + s \ens[a] + k \ens[a]) = \frac{1}{r+s+k} (s \ens[c] + r \ens[a] + k \ens[a])$ and $\frac{1}{s+t+l} (s \ens[c] + t \ens[a] + l \ens[a]) = \frac{1}{s+t+l} (t \ens[d] + s \ens[a] + l \ens[a])$. Let $l' = r - t + k$ and therefore $k = t - r + l'$. We have $\frac{1}{r+s+k} (s \ens[c] + r \ens[a] + k \ens[a]) = \frac{1}{r+s+t-r+l'} (s \ens[c] + r \ens[a] + (t-r +l') \ens[a]) = \frac{1}{s+t+l'} (s \ens[c] + t \ens[a] + l' \ens[a])$. Therefore $l'$ is such that $\frac{1}{s+t+l'} (s \ens[c] + t \ens[a] + l' \ens[a])$ is a valid affine combination, which means$\frac{1}{s+t+l'} (s \ens[c] + t \ens[a] + l' \ens[a]) =\frac{1}{s+t+l'} (t \ens[d] + s \ens[a] + l' \ens[a])$. Now let $k' = l' + s -r$ which gives $l' = k' + r -s$ and $k' = k + s - t$. We have $\frac{1}{r+t+k'} (r \ens[b] + t \ens[a] + k' \ens[a]) = \frac{1}{r+s+k} (r \ens[b] + s \ens[a] + k \ens[a]) = \frac{1}{r+s+k} (s \ens[c] + r \ens[a] + k \ens[a]) = \frac{1}{s+t+l'} (t \ens[d] + s \ens[a] + l' \ens[a]) = \frac{1}{r+t+k'} (t \ens[d] + r \ens[a] + k' \ens[a])$. Therefore $r(\ens[b] - \ens[a]) \sim t(\ens[d] - \ens[a])$.
	\end{proof}
	
	\begin{prop}\label{pm-es-ensDiffProps}
		The ensemble difference equivalence relation satisfies the following:
		\begin{enumerate}
			\item if $r(\ens[b] - \ens[a]) \sim s(\ens[c] - \ens[a])$ with $r\neq0$ and $s\neq0$, then $\ens[a]$, $\ens[b]$ and $\ens[c]$ are on the same line
			\item if $r\neq0$, $[r(\ens[b] - \ens[a])] = \left\{(r+j)\left(\left(\frac{r}{r+j}\ens[b] +\frac{j}{r+j} \ens[a]\right) -\ens[a]\right)\right\}$ for all $j \in \mathbb{R}$ such that the affine combination exists
			\item $\left[0(\ens[b] - \ens[a])\right] = \left\{0\left(\ens - \ens[a]\right) \, | \, \ens \in \Ens \right\} \cup \left\{r\left(\ens[a] - \ens[a]\right) \, | \, r \in \mathbb{R} \right\}$
			\item given $p \in (0,1]$, $r(\ens[b] - \ens[a]) \sim \frac{r}{p}((p\ens[b] + \bar{p}\ens[a]) - \ens[a])$
			\item $r(\ens[b] - \ens[a]) \sim s(\ens[c] - \ens[a])$ implies $(rt)(\ens[b] - \ens[a]) \sim (st)(\ens[c] - \ens[a])$ and therefore scalar multiplication maps equivalence classes to equivalence classes
		\end{enumerate}
	\end{prop}
	
	\begin{proof}
		For 1, the condition for equivalence imposes that an affine combination of $\ens[b]$ and $\ens[a]$ equals an affine combination of $\ens[c]$ and $\ens[a]$. Equivalently, this says that they are expressible as affine combinations of each other, that they are on the same line.
		
		For 2, let us first verify that $r(\ens[b] - \ens[a]) \sim (r+j)\left(\left(\frac{r}{r+j}\ens[b] +\frac{j}{r+j} \ens[a]\right) -\ens[a]\right)$. We have $\frac{1}{r+r+j+k} (r \ens[b] + (r+j) \ens[a] + k \ens[a]) = \frac{1}{r+r+j+k} \left((r+j)\left(\frac{r}{r+j}\ens[b] +\frac{j}{r+j} \ens[a]\right) + r \ens[a] + k \ens[a]\right)=\frac{1}{r+r+j+k} \left(r\ens[b] +j\ens[a] + r \ens[a] + k \ens[a]\right)$, which means the two differences are equivalent. Note that all ensembles that are on the same line of $\ens[a]$ and $\ens[b]$ can, except for $\ens[a]$, be expressed as $\frac{r}{r+j}\ens[b] +\frac{j}{r+j} \ens[a]$. Since, by 1, being on the same line is a necessary condition for equivalence, the expression spans the entire equivalence class.
		
		For 3, suppose $r (\ens[c] - \ens[a]) \sim 0 (\ens[b] - \ens[a])$. Then $\frac{1}{r+0+k} (r \ens[c] + 0 \ens[a] + k \ens[a]) = \frac{1}{r+0+k} (0 \ens[b] + r \ens[a] + k \ens[a])$. If $r=0$ we have $\ens[a] = \ens[a]$ which is satisfied for all $\ens[c] \in \Ens$. If $r\neq0$ we have $\frac{1}{r+k} (r \ens[c] + k \ens[a]) = \ens[a]$ which is satisfied only if $\ens[c] = \ens[a]$.
		
		For 4, let $\frac{r}{p}=r+j$. We have $j = \frac{r}{p}-r = \frac{r(1-p)}{p} = (r+j)\bar{p}$. Therefore $\frac{r}{r+j} = r\frac{p}{r} = p$ and $\frac{j}{r+j} = \bar{p}$. This matches the coefficients in the equivalence class.
		
		For 5, we have
		\begin{equation}
			\begin{aligned}
				\frac{1}{r+s+k} (r \ens[b] + s \ens[a] + k \ens[a]) &= \frac{1}{r+s+k} (s \ens[c] + r \ens[a] + k \ens[a]) \\
				\frac{1}{rt+st+kt} (rt \ens[b] + st \ens[a] + kt \ens[a]) &= \frac{1}{rt+st+kt} (st \ens[c] + rt \ens[a] + kt \ens[a]) \\
				\frac{1}{rt+st+k'} (rt \ens[b] + st \ens[a] + k' \ens[a]) &= \frac{1}{rt+st+k'} (st \ens[c] + rt \ens[a] + k' \ens[a])
			\end{aligned}
		\end{equation}
		which means $(rt)(\ens[b] - \ens[a]) \sim (st)(\ens[c] - \ens[a])$.
	\end{proof}
	
	\begin{defn}
		The \textbf{addition} between two equivalence classes of differences is defined by 
		$$\left[r(\ens[b] - \ens[a])\right] + \left[s(\ens[c] - \ens[a])\right] = \left[(r+s+k) \left(\left(\frac{r}{r+s+k}\ens[b] + \frac{s}{r+s+k} \ens[c] +\frac{k}{r+s+k} \ens[a] \right) - \ens[a]\right)\right]$$
		where $r+s+k \neq 0$.
	\end{defn}
	\begin{check}
		We need to show that the definition does not depend on the chosen representatives of the classes on the left hand side. We first check the case when both classes are of the form $[0(\ens[b] - \ens[a])]$. Suppose we have two elements of said equivalence class. Either the coefficient is zero or the two ensembles are the same. In either case, the elements on the right side of the definition of the addition will be affine combinations of $\ens[a]$ only. Therefore the result of the addition of $[0(\ens[b] - \ens[a])]$ with itself gives us $[0(\ens[b] - \ens[a])]$. Now suppose we sum an equivalence class $[r(\ens[b] - \ens[a])]$ with $[0(\ens[c] - \ens[a])]$. Note that either $s=0$ or $\ens[c]=\ens[a]$. In either case, the ensemble in the result is an affine combination of only $\ens[b]$ and $\ens[a]$ for every $k$. The result, then, spans exactly the equivalence class $[r(\ens[b] - \ens[a])]$. Lastly, suppose we are summing two equivalence classes different from $[0(\ens[b] - \ens[a])]$. The ratio between $\ens[b]$ and $\ens[c]$ remains the same for all $k$, therefore the ensembles in the result span the line between $\ens[a]$ and an affine combination of $\ens[b]$ and $\ens[c]$, which means all elements fall in the same equivalence class.
	\end{check}
	
	\begin{coro}
		Let $\ens[a] \in \Ens$ be an ensemble and let $V = \{ [r(\ens[b]-\ens[a])] \}$ be the set of equivalence classes of ensemble differences from $\ens[a]$. Then the addition of ensemble difference over $V$ is a commutative monoid. If $\ens[a]$ is an internal point, the addition is an abelian group.
	\end{coro}
	
	\begin{proof}
		The operation is commutative as the definition of addition is symmetric over its arguments. The operation is associative as further sums of equivalence classes turn into additions of the real coefficients and extensions of affine combinations both of which are associative. The zero equivalence class $[0(\ens[b] - \ens[a])]$ is the identity element, as shown in the previous proof. The addition is a commutative monoid.
		
		Now let $\ens[a]$ be an internal point and let $[r(\ens[b] - \ens[a])]$ be an equivalence class. Since $\ens[a]$ is an internal point, we can find $\ens[c] \in \Ens$ such that $\ens[a] = p \ens[b] + \bar{p} \ens[c]$. Consider $\left[r\frac{\bar{p}}{p}(\ens[c] - \ens[a])\right]$. Note that $r + r\frac{\bar{p}}{p} = \frac{r}{p}(p+\bar{p}) = \frac{r}{p}$. The addition is given by
		\begin{equation}
			\begin{aligned}
				&\left[\left(r+r\frac{\bar{p}}{p}+k\right) \left(\left(\frac{r}{r+r\frac{\bar{p}}{p}+k}\ens[b] + \frac{r\frac{\bar{p}}{p}}{r+r\frac{\bar{p}}{p}+k} \ens[c] +\frac{k}{r+r\frac{\bar{p}}{p}+k} \ens[a] \right) - \ens[a]\right)\right] \\
				&=\left[\left(\frac{r}{p}+k\right) \left(\left(\frac{\frac{r}{p}}{\frac{r}{p}+k}(p\ens[b] + \bar{p} \ens[c]) +\frac{k}{\frac{r}{p}+k} \ens[a] \right) - \ens[a]\right)\right] \\
				&=\left[\left(\frac{r}{p}+k\right) \left(\left(\frac{\frac{r}{p}}{\frac{r}{p}+k}\ens[a] +\frac{k}{\frac{r}{p}+k} \ens[a] \right) - \ens[a]\right)\right] \\
				&= [0(\ens[b] - \ens[a])].
			\end{aligned}
		\end{equation}
		This means that $\left[r\frac{\bar{p}}{p}(\ens[c] - \ens[a])\right]$ is the inverse of $[r(\ens[b] - \ens[a])]$ and that the addition is an abelian group.
	\end{proof}
	
	\begin{thrm}[Differences form a vector space]
		Let $\ens[a] \in \Ens$ be an internal point and let $V = \{ [r(\ens[b]-\ens[a])] \}$ be the set of equivalence classes of ensemble differences from $\ens[a]$. Then $V$ is a vector space under scalar multiplication and addition.
	\end{thrm}
	
	\begin{proof}
		Scalar multiplication is compatible with the multiplication between scalars since $r(s(t(\ens[b] - \ens[a])))= r((st)(\ens[b]-\ens[a])) = (rst)(\ens[b]-\ens[a]) = (rs)(t(\ens[b]-\ens[a]))$. The identity of the scalars is the identity of scalar multiplication since $1(r(\ens[b]-\ens[a])) = r(\ens[b] - \ens[a])$. Scalar multiplication is distributive with respect to vector addition:
		\begin{equation}
			\begin{aligned}
				&t\left(\left[r(\ens[b] - \ens[a])\right] + \left[s(\ens[c] - \ens[a])\right]\right) \\
				&= t\left[(r+s+k) \left(\left(\frac{r}{r+s+k}\ens[b] + \frac{s}{r+s+k} \ens[c] +\frac{k}{r+s+k} \ens[a] \right) - \ens[a]\right)\right] \\
				&= \left[t(r+s+k) \left(\left(\frac{r}{r+s+k}\ens[b] + \frac{s}{r+s+k} \ens[c] +\frac{k}{r+s+k} \ens[a] \right) - \ens[a]\right)\right] \\
				&= \left[(tr+ts+tk) \left(\left(\frac{tr}{tr+ts+tk}\ens[b] + \frac{ts}{tr+ts+tk} \ens[c] +\frac{tk}{tr+ts+tk} \ens[a] \right) - \ens[a]\right)\right] \\
				&= \left[(tr+ts+k') \left(\left(\frac{tr}{tr+ts+k'}\ens[b] + \frac{ts}{tr+ts+k'} \ens[c] +\frac{k'}{tr+ts+k'} \ens[a] \right) - \ens[a]\right)\right] \\
				&\left[tr(\ens[b] - \ens[a])\right] + \left[ts(\ens[c] - \ens[a])\right].
			\end{aligned}
		\end{equation}
		Scalar multiplication is distributive with respect to scalar addition
		\begin{equation}
			\begin{aligned}
				&s\left[r(\ens[b] - \ens[a])\right] + t\left[r(\ens[b] - \ens[a])\right] \\
				&\left[sr(\ens[b] - \ens[a])\right] + \left[tr(\ens[b] - \ens[a])\right] \\
				&= \left[(sr+tr+k) \left(\left(\frac{sr}{sr+tr+k}\ens[b] + \frac{tr}{sr+tr+k} \ens[b] +\frac{k}{sr+tr+k} \ens[a] \right) - \ens[a]\right)\right] \\
				&= \left[(sr+tr+k) \left(\left(\frac{sr+tr}{sr+tr+k}\ens[b] + \frac{k}{sr+tr+k} \ens[a] \right) - \ens[a]\right)\right] \\
				&= [((s+t)r)(\ens[b]-\ens[a])] \\
				&= (s+t)[r(\ens[b]-\ens[a])].
			\end{aligned}
		\end{equation}
		Therefore $V$ with addition and scalar multiplication satisfies the definition of a vector space.
	\end{proof}
	
	\begin{defn}
		Given an internal point $\ens[a]$, the \textbf{natural embedding} of $\Ens$ into $V_{\ens[a]}$ is the map $\iota_{\ens[a]} : \Ens \hookrightarrow V_{\ens[a]}$, defined as $\iota_{\ens[a]}(\ens) \to [1(\ens - \ens[a])]$, that maps each ensemble to its difference from $\ens[a]$.
	\end{defn}
	
	\begin{coro}
		The natural embedding of $\Ens$ into $V_{\ens[a]}$ preserves affine combinations. That is, $\iota\left(\sum_{i=1}^n r_i \ens_i \right) = \sum_{i=1}^n r_i \iota(\ens_i)$.
	\end{coro}
	
	\begin{proof}
		Let $r+s=1$ and $\ens[b], \ens[c] \in \Ens$ such that $r\ens[b] + s \ens[c] \in \Ens$. We have
		\begin{equation}
			\begin{aligned}
				r[1(\ens[b] - \ens[a])] +s[1(\ens[c] - \ens[a])] &= [r(\ens[b] - \ens[a])] +[s(\ens[c] - \ens[a])] \\
				&= \left[(r+s+k) \left(\left(\frac{r}{r+s+k}\ens[b] + \frac{s}{r+s+k} \ens[c] +\frac{k}{r+s+k} \ens[a] \right) - \ens[a]\right)\right] \\
				&= \left[(1+k) \left(\left(\frac{r}{1+k}\ens[b] + \frac{s}{1+k} \ens[c] +\frac{k}{1+k} \ens[a] \right) - \ens[a]\right)\right] \\
				&= \left[1 \left(\left(\frac{r}{1}\ens[b] + \frac{s}{1} \ens[c] \right) - \ens[a]\right)\right] \\
				&= \left[1\left(\left(r\ens[b] + s \ens[c] \right) - \ens[a]\right)\right].
			\end{aligned}
		\end{equation}
		If we apply the above recursively, we have $\sum_{i=1}^{n} r_i [1(\ens_i - \ens[a])] = \left[1\left(\left(\sum_{i=1}^{n}r_i \ens_i\right) - \ens[a]\right)\right]$. Therefore $\sum_{i=1}^n r_i \iota(\ens_i) = \iota\left(\sum_{i=1}^n r_i \ens_i \right)$.
	\end{proof}
	
	\begin{prop}
		Let $\ens[a],\ens[b] \in \Ens$ be two internal points. Then $r(\ens[c] - \ens[b]) \mapsto r(\ens[c]-\ens[a]) - r(\ens[b]-\ens[a])$ is an isomorphism between the corresponding vector spaces.
	\end{prop}
	
	\begin{proof}
		Note that the natural embeddings of $\Ens$ in $V_{\ens[a]}$ and $V_{\ens[b]}$ are invertible onto their images, and therefore they give an affine map between $V_{\ens[a]}$ and $V_{\ens[b]}$. That is, $r(\ens[c] - \ens[b]) \mapsto r(\ens[c]-\ens[a])$ is an affine map. Moreover, $r(\ens[c] - \ens[b]) \mapsto r(\ens[c]-\ens[a]) + v$ will be an affine map for any $v \in V_{\ens[a]}$. An affine map is linear if and only if it maps the zero vector to the zero vector. Note that $r(\ens[b] - \ens[b])$ is the zero vector for $V_{\ens[b]}$, which means the above map is affine if and only if $r(\ens[b] - \ens[b]) \mapsto r(\ens[a] - \ens[a]) \sim r(\ens[b]-\ens[a]) - r (\ens[b]-\ens[a])$ and therefore $v = -r(\ens[b]-\ens[a])$.
	\end{proof}
\end{mathSection}

We leave open whether the ensemble space embeds continuously in a topological vector space, and whether the topological vector space would have to be a locally convex. The first step would be to check whether this is true in the finite-dimensional case.

%\begin{mathSection}
%\begin{conj}\label{pm-es-ensemblesAreTVS}
%	An ensemble space embeds continuously in a topological vector space.
%\end{conj}

%\begin{remark}
%	Axioms of ensemble and mixture are not enough to guarantee the topological nature of the vector space. See \href{https://math.stackexchange.com/questions/4921905/cancellative-convex-spaces-and-topological-vector-spaces}{this post}. The issue is that we have no guarantee of continuity on the inverse (i.e. subtraction, multiplication by a negative number). As it is shown later, the entropy provides a pseudo-distance, which allows one to create open balls. Moreover, the entropy induces a metric tensor, which may induce a suitable topology.
%\end{remark}
%\end{mathSection}

\subsubsection{Self-mixtures}

As an example of a convex space that is non-cancellative, does not embed in a vector space, and therefore is ruled out as an ensemble space, we consider one in which the convex combination of two different elements returns the first. We show that this can be made to satisfy the axioms of ensemble and mixture, and therefore it is really the axiom of entropy that rules it out.

\begin{example}[Self-mixture]
	Let $\Ens = \{\ens[a], \ens[b]\}$ be a set endowed with the convex structure that satisfies identity, idempotence, commutativity and such that $p \ens[a] + \bar{p} \ens[b]$ equals $\ens[a]$ if $p=1$ and $\ens[b]$ otherwise. Identity, idempotence and commutativity are satisfied by construction. For associativity, note that the final result of multiple mixtures is $\ens[a]$ if and only if all elements of the mixtures are $\ens[a]$. The order does not matter, and therefore associativity is satisfied. Therefore such structure satisfies at least the axiom of mixture without the requirement of continuity.
	
	For continuity, we need to look at the inverse image under the mixing operation. Note that $+^{-1}(\emptyset) = \emptyset$ and $+^{-1}(\Ens) = [0,1] \times \Ens \times \Ens$. Since $\emptyset$ and $\Ens$ must be in any topology of $\Ens$, the continuity condition is satisfied for these sets. Next, we have  $+^{-1}(\{\ens[a]\}) = [0,1] \times \{\ens[a]\} \times \{\ens[a]\} \cup [1] \times \{\ens[a]\} \times \{\ens[b]\} \cup [0] \times \{\ens[b]\} \times \{\ens[a]\}$. Note that $[0]$ and $[1]$ are not open sets, therefore $\{\ens[a]\}$ cannot be in the topology of $\Ens$ or mixing would not be continuous. Finally, $+^{-1}(\{\ens[b]\}) = [0,1] \times \{\ens[b]\} \times \{\ens[b]\} \cup [0,1) \times \{\ens[a]\} \times \{\ens[b]\} \cup (0,1] \times \{\ens[b]\} \times \{\ens[a]\} = [0,1) \times \Ens \times \{\ens[b]\} \cup (0,1] \times \{\ens[b]\} \times \Ens$. Since $(0,1]$ and $[0,1)$ are open subsets of $[0,1]$, \{$\ens[b]\}$ can be an open set. Since the topology must be $\textsf{T}_0$, we must include at least $\{\ens[b]\}$ and therefore $\mathsf{T}_{\Ens} = \{\emptyset, \{\ens[b]\}, \{\ens[a], \ens[b]\}\}$ is a $\mathsf{T}_0$ second countable topology for which mixing is continuous. This means that the example satisfies both the axioms of ensemble and of mixture.
\end{example}

It is therefore the entropy that rules out this case. If the entropy of the two ensembles is different, it would jump from $S(\ens[a])$ directly to $S(\ens[b])$ for an infinitesimal mixture which violates the upper bound. If $S(\ens[a])$ equals $S(\ens[b])$, the entropy of the mixture of $\ens[a]$ and $\ens[b]$ is also the same, so the two ensembles cannot possibly be different by strict concavity.

Intuitively, this tells us that we cannot have a mixture of two different elements that happens to be equal to one of the original elements.

\begin{mathSection}
	\begin{coro}
		Let $\ens[a], \ens[b] \in \Ens$ such that $p\ens[a] + \bar{p} \ens[b] = \ens[b]$ for some $p \in (0,1]$. Then $\ens[a] = \ens[b]$. 
	\end{coro}
	
	\begin{proof}
		We have $p\ens[a] + \bar{p} \ens[b] = \ens[b] = p\ens[b] + \bar{p} \ens[b]$ for some $p \in (0,1]$. Therefore $\ens[a] = \ens[b]$.
	\end{proof}
\end{mathSection}

\subsection{Boundedness of lines}

Another constraint that the entropy imposes is that the ensemble space is bounded along every direction. That is, if we take two ensembles, these will identify an affine line in the embedding vector space that will, at some point, exit the ensemble space in both directions.

\begin{figure}[h]
	\centering
	\begin{tikzpicture}
		%\draw [help lines] (0,0) grid (10,3);
		\coordinate (O) at (1, 1);
		\coordinate (x) at (0, 0);
		\coordinate (y) at (3, 1);
		\coordinate (z) at (1, 3);
		
		\node[below left] at (z) {$V_{\ens}$};
		\node[above] at (2,2.6) {$\Ens$};
		\node[below right] at (O) {$\ens$};
		
		\draw[-] (O) -- (x);
		\draw[-] (O) -- (y);
		\draw[-] (O) -- (z);
		
		\draw[-, thick] plot [smooth cycle] coordinates {(0.5,1.5)(1.5,0.3)(2.7,1)(2.6,2.5)(1,2.5)};
		
		\draw[dashed] (2, 2) -- (3, 2.5);
		\draw[-, thick] (1.5, 1.75) -- (2, 2) node[below] {$b$};
		\draw[dashed] (0.4,1.2) -- (1.5, 1.75) node[below] {$a$};
	\end{tikzpicture}
	%\includegraphics[width=0.4\textwidth]{tempimages/BoundedEnsembleSpace.jpg}
	\caption{$\Ens$ is the ensemble space and $V_{\ens}$ is the embedding vector space with origin $\ens$. Any affine line, like the dashed one in the picture, will intersect $\Ens$ over a finite range limited by the two elements $a$ and $b$ of the vector space (but not necessarily of the ensemble space).}
\end{figure}

The proof proceeds as follows. Suppose we take three points and assign them an entropy value consistent with the entropic bounds: can we always pick an entropy value that satisfies the bounds for any subsequent point in a given direction? The answer is negative and figure \ref{pm-es-directionallyBound} represents the problem pictorially. On the horizontal axis we can imagine all the ensembles over a line, and on the vertical axis their entropy. We fix the entropy for the origin, the blue and the red point. What are the values for the next point, the green one, that satisfy the entropy bounds? Because of strict concavity, the green point must remain below the blue line. The green line represents the upper bound for the blue point, which means the green point must be above the purple line. If we go far enough, the blue and purple line meet, leaving no possible value for the entropy.

\begin{figure}[H]
	\centering
	\begin{tikzpicture}
		%\draw [help lines] (0,0) grid (4,4);
		\def\ax{0}
		\def\bx{1.9}
		\def\cx{3.8}
		\def\dx{5.7}
		
		\def\aS{0}
		\def\bS{1}
		\def\cS{1.5}
		\def\dS{1.55}
		
		\def\xmin{-1}
		\def\xmax{21.5}
		
		\begin{axis}[
			height=5.5cm,
			width=14cm,
			axis lines=middle,
			ylabel={$S$},
			xlabel={$x$},
			xlabel style={below left},
			xmin=\xmin,xmax=\xmax,
			ymin=-0.5,ymax=8.5
			]
			\addplot[only marks, black] coordinates { (\ax,\aS) };
			\addplot[only marks, blue!75!black] coordinates { (\bx,\bS) };
			\addplot[only marks, red!75!black] coordinates { (\cx,\cS) };
			\addplot[only marks, green!75!black] coordinates { (\dx,\dS) };
			\addplot[blue!75!black,samples=10,domain=\xmin:\xmax]{(x - \bx)/(\cx - \bx)*(\cS - \bS) + \bS};
			\addplot[green!75!black,samples=50,domain=0:\dx]{-(x/\dx) * ln((x/\dx))/ln 2 - (1-(x/\dx)) * ln(1-(x/\dx))/ln 2 + x / \dx * \dS};
			\addplot[blue!60!red,samples=300,domain=\bx:\xmax]{x/\bx*(\bS-(-(\bx/x) * ln((\bx/x))/ln 2 - (1-(\bx/x)) * ln(1-(\bx/x))/ln 2))};
		\end{axis}
	\end{tikzpicture}
	%\includegraphics[width=0.7\textwidth]{tempimages/BoundsFromEntropy.png}
	\caption{Visual representation of the bounds. See proof for details.}\label{pm-es-directionallyBound}
\end{figure}

\begin{mathSection}
\begin{defn}
	A \textbf{line} $A \subseteq \Ens$ is a convex subset such that for any three elements one can be expressed as a mixture of the other two. That is, for all distinct $\ens_1, \ens_2, \ens_3 \in A$ there exists a permutation $\sigma : \{1,2,3\} \to \{1,2,3\}$ and $p \in (0,1)$ such that $\ens_{\sigma(1)} = p \ens_{\sigma(2)} + \bar{p} \ens_{\sigma(3)}$.
\end{defn}

\begin{prop}
	Given two distinct $\ens[a], \ens[b] \in \Ens$, there is only one line that contains them.
\end{prop}

\begin{proof}
	Since an ensemble space embeds into a vector space, given two distinct ensembles there is only one affine line that connects them. A line is the intersection of the affine line with the ensemble space.
\end{proof}

\begin{thrm}[Lines are bounded]
	Let $A \subseteq \Ens$ be a line. Then we can find a bounded interval $V \subset \mathbb{R}$ and an invertible function $f : A \to V$ such that $f(p \ens[a] + \bar{p} \ens[b]) = p f(\ens[a]) + \bar{p} f(\ens[b])$ for all $\ens[a], \ens[b] \in A$.
\end{thrm}

\begin{proof}
	Let $A \subseteq \Ens$ be a line. Pick $\ens_0, \ens_1 \in A$. For any $\ens[a] \in A$, we can write it as an affine combination of $\ens_0$ and  $\ens_1$. That is, $\ens[a] = \bar{x}\ens_0 + x\ens_1$ where $x \in \mathbb{R}$. Note that $\ens[a]$ uniquely determines $x$. Therefore we can define $f(\ens[a]) \mapsto x$, and it will be an invertible function.
	
	We can verify that, given $\ens[a], \ens[b] \in A$, we have
	\begin{equation}
		\begin{aligned}
			f(p\ens[a] + \bar{p} \ens[b]) &= f(p\bar{x}_a \ens_0 + px_a \ens_1 + \bar{p}\bar{x}_b \ens_0 + \bar{p}x_b \ens_1) = f((p\bar{x}_a + \bar{p}\bar{x}_b) \ens_0 + (px_a + \bar{p}x_b) \ens_1) \\
			&= px_a + \bar{p}x_b = pf(\ens[a]) + \bar{p}f(\ens[b]).
		\end{aligned}
	\end{equation}
	
	We are going to show that the image $V=f(A)$ must be a bounded set, or it would eventually violate the entropy bounds. Given a value $x \in V$, there will be an entropy value $S(f^{-1}(x))$ which we can write as a function of the real value $S(x)$. This function will need to satisfy continuity, strict concavity and the upper variability bound.

	We are going to show that the function $S(x)$ cannot extend to plus infinity. The same argument can be applied by symmetry for minus infinity. We are going to assume that $S(0) = 0$ without loss of generality. If $S$ is not defined at zero, or if the value is different, we can apply a translation on the argument or on the value, which will not affect the concavity of the function.
	
	Let $0 < a < b \in V$ be two distinct values, with $S(a)$ and $S(b)$ their respective entropies. We are going to show that if we pick a $c > b$ sufficiently large, we are not going to find a value for $S(c)$ that satisfies the bounds. In the picture, we have the three points, $a$ in blue, $b$ in red, $c$ in green. The horizontal axis represents the value, the position of the ensemble along the affine line, while the vertical axis represents the entropy. Consider the points $a$, $b$ and $c$. Since the entropy is strictly concave, $c$ must be placed under the blue line which represents an upper bound on $S(c)$. Now consider $0$, $a$ and $c$. Since $a$ is a mixture of $0$ and $c$, $S(a)$ will need to satisfy the upper bound given by $S(0)$ and $S(c)$. In the diagram, the green line represents the upper bound on $S(a)$ for the specific choice of $c$ and $S(c)$. Since $a$ and $S(a)$ are fixed, this puts a lower bound on $S(c)$, which is represented by the purple curve. That is, the purple curve represents the minimum value we have to assign to $S(c)$ such that $S(a)$ still satisfies the upper bound between $0$ and $c$. As the picture shows, the two bounds meet at some point and cannot both be satisfied.
	
	From strict concavity, noting that $\frac{c-b}{c-a} + \frac{b-a}{c-a} = 1$, we have:
	\begin{equation}
		\begin{aligned}
			S(b) &= S\left(\frac{c-b}{c-a} a + \frac{b-a}{c-a} c\right) > \frac{c-b}{c-a} S(a) + \frac{b-a}{c-a} S(c)\\
			(b-a)S(c) &< (c-a) S(b) - (c-b) S(a) = (c-a) S(b) - (c-a) S(a) + (b-a) S(a) \\
			S(c) &< \frac{S(b) - S(a)}{(b-a)} (c-a) + S(a) \\
		\end{aligned}
	\end{equation}
	
	From the upper bound, noting that $\frac{c-a}{c} + \frac{a}{c} = 1$ and recalling we assumed $S(0) = 0$, we have:
	\begin{equation}
		\begin{aligned}
			S(a) &= S\left(\frac{c-a}{c} 0 + \frac{a}{c} c\right) \leq I\left(\frac{c-a}{c}, \frac{a}{c}\right) + \frac{c-a}{c} S(0) +\frac{a}{c} S(c) \\
			\frac{a}{c} S(c) &\geq S(a) - I\left(\frac{c-a}{c}, \frac{a}{c}\right) \\
			S(c) &\geq \frac{c}{a} \left[ S(a) - I\left(\frac{c-a}{c}, \frac{a}{c}\right) \right]
		\end{aligned}
	\end{equation}
	
	Combining the bounds, we have:
	\begin{equation}
		\begin{aligned}
			\frac{S(b) - S(a)}{(b-a)} (c-a) + S(a) &> \frac{c}{a} \left[ S(a) - I\left(\frac{c-a}{c}, \frac{a}{c}\right) \right] \\
			(c-a) S(b) - (c-a)S(a) + (b-a) S(a) &> \frac{c(b-a)}{a} S(a) - \frac{c(b-a)}{a} I\left(\frac{c-a}{c}, \frac{a}{c}\right) \\
			a (c-a) S(b) - a(c-b) S(a) &> c(b-a)S(a) - c(b-a) I\left(\frac{c-a}{c}, \frac{a}{c}\right) \\
			a (c-a) S(b) + (-ac +ab -bc +ac) S(a) &> - c(b-a) I\left(\frac{c-a}{c}, \frac{a}{c}\right) \\
			a (c-a) S(b) -b (c-a) S(a) &> - c(b-a) I\left(\frac{c-a}{c}, \frac{a}{c}\right) \\
			a  S(b) - b S(a) &> - \frac{c(b-a)}{(c-a)} I\left(\frac{c-a}{c}, \frac{a}{c}\right) \\
		\end{aligned}
	\end{equation}
	
	Since $b > a$, $c > a$ and $I(p,\bar{p}) > 0$ for all $p \in (0,1)$, the right hand side of the inequality is always negative. As $c$ increases, the right hand side will go to zero, since $\lim\limits_{c\to \infty}\frac{c(b-a)}{c-a} = b-a$ and $\lim\limits_{c\to \infty} I\left(\frac{c-a}{c}, \frac{a}{c}\right) = I(1,0) = 0$. The left hand side is a constant. If the constant is positive, the inequality is always satisfied. If it is negative, it will not be satisfied for all $c$.
	
	From strict concavity, noting that $\frac{b-a}{b} + \frac{a}{b} = 1$, we have
	\begin{equation}
		\begin{aligned}
			S(a) &= S\left(\frac{b-a}{b} 0 + \frac{a}{b} b\right) > \frac{b-a}{b} S(0) + \frac{a}{b} S(b) = \frac{a}{b} S(b)\\
			b S(a) &> a S(b)  \\
			0 &> a S(b) - b S(a)\\
		\end{aligned}
	\end{equation}
	This shows that the left hand side of the previous inequality is negative, and therefore the bounds cannot be satisfied over the whole $\mathbb{R}$.
	
	This means that $V = f(A)$ must be bounded and therefore every line is a segment as embedded in the vector space.
\end{proof}

\begin{remark}
	Note that this does not mean that, along each direction, the line is closed in the vector space. That is, it may not include the extreme points in the convex space. An open bounded interval, in fact, is still a convex space and we would be able to define an entropy on it.
\end{remark}
\end{mathSection}

The fact that the ensemble space is directionally bounded gives us an intuitive property that we would, mistakenly, always think to be true. Suppose we have an ensemble $\ens$ and a sequence of ensembles $\ens[a]_i \in L$ on some line $L$ that contains $\ens$. Then $p_i \ens + \bar{p}_i \ens[a]_i \to \ens$ if $p_i \to 1$. This does not work in a generic convex space if directions are unbounded. For example, suppose $\ens = \mathbb{R}$, which is a convex set but not directionally bounded. Let $\ens = 0$, $\ens[a]_i = i$ and $p_i = 1 - \frac{1}{i}$. Then $\left(1 - \frac{1}{i}\right) 0 + \frac{1}{i}{i} = 1$. Therefore $p_i \ens + \bar{p}_i \ens[a]_i \nto \ens$ even though $p_i \to 1$. It is, again, the entropy that guarantees that this intuitive property is satisfied.

\begin{mathSection}
	\begin{prop}
		Let $L \subseteq \Ens$ be a line. Let $\ens \in L$ and $\ens[a]_i \in L$. Then $p_i \ens + \bar{p}_i \ens[a]_i \to \ens$ if $p_i \to 1$.
	\end{prop}
	
	\begin{proof}
		Pick $\ens[a] \in L$ such that $\ens[a] \neq \ens$. Then every $\ens[a]_i$ can be expressed as an affine combination $\ens[a]_i = r_i \ens + \bar{r}_i \ens[a]$. We have $p_i \ens + \bar{p}_i \ens[a]_i = p_i \ens + \bar{p}_i \left( r_i \ens + \bar{r}_i \ens[a] \right) = \left( p_i + \bar{p}_i r_i \right) \ens + \bar{p}_i\bar{r}_i \ens[a]$. Since the ensemble space is directionally bounded, the sequence $r_i$ is bounded from below and above. This means that $\bar{p}_i \to 0$ implies $p_i + \bar{p}_i r_i \to 1$ and $\bar{p}_i\bar{r}_i \to 0$. Therefore by continuity $\left( p_i + \bar{p}_i r_i \right) \ens + \bar{p}_i\bar{r}_i \ens[a] \to 1 \ens + 0 \ens[a] = \ens$.
	\end{proof}
\end{mathSection}

It is an open question whether this property is guaranteed for a generic sequence of ensembles $\ens[a]_i$, not necessarily on a line. At least, it should be extensible to the finite-dimensional case.

\begin{conj}
	Let $\ens \in \Ens$ and $\ens[a]_i \in \Ens$. Then $p_i \ens + \bar{p}_i \ens[a]_i \to \ens$ if $p_i \to 1$.
\end{conj}


As discussed previously, it may be that the boundedness provided by the entropy forces all submixtures of convergent infinite mixtures to converge.

\begin{conj}\label{pm-es-conjSubmixturesConverge}
	Let $\ens[a] = \sum_{i=1}^{\infty} p_i \ens_i$. Then, given $I \subseteq \mathbb{N}$, $\sum_{i \in I} \frac{p_i}{p_I} \ens_i$ converges where $p_I = \sum_{i \in I} p_i$.
\end{conj}

\begin{conj}\label{pm-es-conjAlgebraicInteriorSigmaConvex}
	The set of all internal points $I_{\Ens}$ is closed under infinite convex combinations.
\end{conj}


%TODO: for topology, the ensemble space is a convex neighbourhood of the origin, which means it is an absorbing set.

%\begin{prop}
%	The ensemble space is, as a subset of the embedding vector space, a radial set and, since the vector space is real, an absorbing set.
%\end{prop}

%\begin{proof}
%	Let $\ens[a] \in \Ens$ be an internal point and $\Ens \subseteq V_{\ens[a]}$ be the ensemble space as a subset of the embedding vector space. Since $\ens[a]$ is an internal point, for every direction there exist a finite segment that starts from $\ens[a]$ and proceeds in that direction. Therefore the ensemble space is a radial set. Moreover, since in a real vector space \href{https://en.wikipedia.org/wiki/Radial_set#Relation_to_absorbing_sets}{every radial is absorbing}, the ensemble space is an absorbing set. TODO: find a better reference.
%\end{proof}

\section{Entropic geometry}

In this section we will see how the entropy imposes a geometric structure on the ensemble space. The core idea is that, since the entropy is strictly concave, its Hessian is negative definite. The negation is therefore a positive definite, real-valued function of two variations and plays the role of a metric tensor.

\subsection{Mixing entropy as pseudo-distance}

The first observation is that the entropy can be used to define a pseudo-distance. If we mix two ensembles, in fact, the average entropy cannot decrease and will stay the same if the two components of the mixture are the same ensemble. The increase, then, is zero if the two ensembles are equal, greater than zero if not, with the maximum if they are orthogonal. This means that the increase of entropy during mixing can be used to characterize how different two ensembles are.

We define the mixing entropy as the increase in entropy for the equal mixture of two states. This satisfies all axioms for a distance, except the triangle inequality. The mixing entropy recovers the Jensen-Shannon divergence in both the classical and quantum case, and can therefore be seen as its generalization.

\begin{mathSection}
\begin{defn}
	Given two ensembles $\ens[a], \ens[b] \in \Ens$, the \textbf{mixing entropy}, also called Jensen-Shannon divergence, is the increase in entropy associated to their equal mixture. That is:
	$$MS(\ens[a], \ens[b]) = S\left(\frac{1}{2}\ens[a] + \frac{1}{2} \ens[b]\right) - \left(\frac{1}{2} S(\ens[a]) + \frac{1}{2} S(\ens[b])\right).$$
\end{defn}

\begin{prop}
	The mixing entropy $MS(\ens[a], \ens[b])$ satisfies the following:
	\begin{enumerate}
		\item \textbf{non-negativity}: $MS(\ens[a], \ens[b]) \geq 0$
		\item \textbf{identity of indiscernibles}: $MS(\ens[a], \ens[b]) = 0 \iff \ens[a]=\ens[b]$
		\item \textbf{unit boundedness}: $MS(\ens[a], \ens[b]) \leq 1$
		\item \textbf{maximality of orthogonals}: $MS(\ens[a], \ens[b]) = 1 \iff \ens[a] \ortho \ens[b]$
		\item \textbf{symmetry}: $MS(\ens[a], \ens[b]) = MS(\ens[b], \ens[a])$
	\end{enumerate}
\end{prop}

\begin{proof}
	For 1, by strict concavity, $S\left(\frac{1}{2} \ens[a] + \frac{1}{2} \ens[b]\right) \geq \frac{1}{2} S(\ens[a]) + \frac{1}{2} S(\ens[b])$, which means $S\left(\frac{1}{2} \ens[a] + \frac{1}{2} \ens[b]\right) - \left(\frac{1}{2} S(\ens[a]) + \frac{1}{2} S(\ens[b])\right) = MS(\ens[a], \ens[b]) \geq 0$.
	
	For 2, the concavity is strict and therefore the equality holds if and only if $\ens[a] = \ens[b]$.
	
	For 3, by the upper variability bound, $S\left(\frac{1}{2} \ens[a] + \frac{1}{2} \ens[b]\right) \leq I(\frac{1}{2}, \frac{1}{2}) + \frac{1}{2} S(\ens[a]) + \frac{1}{2} S(\ens[b])$, which means $S\left(\frac{1}{2} \ens[a] + \frac{1}{2} \ens[b]\right) - \left(\frac{1}{2} S(\ens[a]) + \frac{1}{2} S(\ens[b])\right) = MS(\ens[a], \ens[b]) \leq I(\frac{1}{2}, \frac{1}{2}) = 1$.
	
	For 4, the upper variability bound is saturated if and only if $\ens[a] \ortho \ens[b]$.
	
	For 5, by commutativity of mixing and of addition the definition of the mixing entropy is symmetric.
\end{proof}

\begin{prop}
	In discrete and continuous classical cases, the mixing entropy coincides with the Jensen-Shannon divergence. In quantum spaces it coincindes with the quantum Jensen-Shannon divergence.
\end{prop}

\begin{proof}
	Looking at the definitions of the \href{https://en.wikipedia.org/wiki/Jensen%E2%80%93Shannon_divergence}{Jensen-Shannon divergence}, in the classical case we have
	$$ JSD(\ens[a], \ens[b]) = S\left(\frac{1}{2}\ens[a] + \frac{1}{2}\ens[b] \right)  - \frac{1}{2} \left(S(\ens[a]) + S(\ens[b])\right) = MS(\ens[a], \ens[b]),$$
	and, similarly, in the quantum case we have
	$$ QJSD(\ens[a], \ens[b]) = S\left(\frac{1}{2}\ens[a] + \frac{1}{2}\ens[b] \right)  - \frac{1}{2} \left(S(\ens[a]) + S(\ens[b])\right) = MS(\ens[a], \ens[b]).$$
\end{proof}

\begin{remark}
	The mixing entropy fails to be a distance function as it does not satisfy the triangle inequality. In the classical and quantum case, in fact, the JSD and QJSD are the square of a distance function. Given the current axiom, it is unlikely that this result can be generalized to the mixing entropy itself. The issue is that we do not have enough information about the relative entropy of 3 points, apart from the orthogonal case. It is very likely that, on physics grounds, there should be constraints on the mixing entropy between three points.
\end{remark}
\end{mathSection}

Though the mixing entropy is not a distance function, we can still show that, like a distance, it decreases as one ensemble approaches another. The notion of two ensembles approaching is given by the convex structure: in a convex space, a mixture of $\ens[a]$ and $\ens[b]$ is closer to $\ens[a]$ than $\ens[b]$ is to $\ens[a]$.

\begin{mathSection}
\begin{prop}\label{pm-es-mixingEntropyMonotonic}
	Let $\ens[a], \ens[b] \in \Ens$. Then $MS(\ens[a], \ens[b]) \geq MS(\ens[a], p\ens[a]+\bar{p}\ens[b])$ with the equality holding if and only if $p=0$ or $\ens[a] = \ens[b]$.
\end{prop}

\begin{proof}
	First let us show that $\frac{1}{2} \ens[a] + \frac{1}{2} \ens[b]$ can be expressed as a mixture of $\frac{1}{2} \ens[a] + \frac{1}{2}\left(p\ens[a] + \bar{p} \ens[b]\right)$ and $p\ens[a] + \bar{p} \ens[b]$.
	\begin{equation}
		\begin{aligned}
			\frac{1}{2} \ens[a] + \frac{1}{2} \ens[b] &= \frac{1}{2} \frac{1-2p + p - 2p^2 + 2p^2}{1-p} \ens[a] + \frac{1}{2}\left(1-2p+2p\right) \ens[b] \\
			&= \left( \frac{1-2p}{1-p} \frac{1+p}{2} + \frac{2}{2} \frac{p^2}{1-p} \right) \ens[a] + \left( \frac{1-2p}{1-p} \frac{\bar{p}}{2}+ \frac{2}{2} \frac{p\bar{p}}{1-p} \right) \ens[b] \\
			&=  \frac{1-2p}{1-p}\left[\frac{1}{2} \ens[a] + \frac{1}{2}\left(p\ens[a] + \bar{p} \ens[b]\right)\right] + \frac{p}{1-p}\left[p\ens[a] + \bar{p} \ens[b]\right]
		\end{aligned}
	\end{equation}
	From strict concavity we have
	\begin{equation}
		\begin{aligned}
			S\left(\frac{1}{2} \ens[a] + \frac{1}{2} \ens[b] \right) &\geq  \frac{1-2p}{1-p}S\left(\frac{1}{2} \ens[a] + \frac{1}{2}\left(p\ens[a] + \bar{p} \ens[b]\right)\right) + \frac{p}{1-p} S\left(p\ens[a] + \bar{p} \ens[b]\right).
		\end{aligned}
	\end{equation}
	Note that the equality holds only if $p=0$, in which case the second mixing coefficient is zero.
	
	Now let us show that $p \ens[a] + \bar{p} \ens[b]$ can be expressed as a mixture of $\frac{1}{2} \ens[a] + \frac{1}{2}\left(p\ens[a] + \bar{p} \ens[b]\right)$ and $\ens[b]$.
	\begin{equation}
		\begin{aligned}
			p \ens[a] + \bar{p} \ens[b]
			&= p \ens[a] + \bar{p} \frac{p+1}{1+p}\ens[b] \\
			&= \frac{2p}{1+p} \frac{1+p}{2} \ens[a] + \left( \frac{2p}{1+p} \frac{\bar{p}}{2}+ \frac{\bar{p}}{1+p} \right) \ens[b] \\
			&= \frac{2p}{1+p}\left[\frac{1}{2} \ens[a] + \frac{1}{2}\left(p\ens[a] + \bar{p} \ens[b]\right)\right] + \frac{1-p}{1+p} \ens[b]
		\end{aligned}
	\end{equation}
	From strict concavity we have
	\begin{equation}
		\begin{aligned}
			S\left(p \ens[a] + \bar{p} \ens[b] \right)&\geq \frac{2p}{1+p} S\left(\frac{1}{2} \ens[a] + \frac{1}{2}\left(p\ens[a] + \bar{p} \ens[b]\right)\right) + \frac{1-p}{1+p} S(\ens[b]) \\
			-S(\ens[b]) &\geq \frac{1+p}{1-p} \frac{2p}{1+p} S\left(\frac{1}{2} \ens[a] + \frac{1}{2}\left(p\ens[a] + \bar{p} \ens[b]\right)\right) - \frac{1+p}{1-p} S\left(p \ens[a] + \bar{p} \ens[b] \right) \\
			&= \frac{2p}{1-p} S\left(\frac{1}{2} \ens[a] + \frac{1}{2}\left(p\ens[a] + \bar{p} \ens[b]\right)\right) - \frac{1+p}{1-p} S\left(p \ens[a] + \bar{p} \ens[b] \right).
		\end{aligned}
	\end{equation}
	Note that the equality holds only if $p=0$, in which case the first mixing coefficient is zero.
	
	Putting it all together
	\begin{equation}
		\begin{aligned}
			MS(\ens[a],\ens[b]) &= S\left(\frac{1}{2} \ens[a] + \frac{1}{2} \ens[b]\right)- \frac{1}{2} S(\ens[a]) - \frac{1}{2} S(\ens[b]) \\
			&\geq \frac{1-2p}{1-p}S\left(\frac{1}{2} \ens[a] + \frac{1}{2}\left(p\ens[a] + \bar{p} \ens[b]\right)\right) + \frac{p}{1-p} S\left(p\ens[a] + \bar{p} \ens[b]\right) - \frac{1}{2} S(\ens[a]) \\
			&+ \frac{1}{2}\left[\frac{2p}{1-p} S\left(\frac{1}{2} \ens[a] + \frac{1}{2}\left(p\ens[a] + \bar{p} \ens[b]\right)\right) - \frac{1+p}{1-p} S\left(p \ens[a] + \bar{p} \ens[b] \right)\right] \\
			&= \left(\frac{1-2p + p}{1-p}\right)S\left(\frac{1}{2} \ens[a] + \frac{1}{2}\left(p\ens[a] + \bar{p} \ens[b]\right)\right) - \frac{1}{2} S(\ens[a]) + \frac{1}{2}\left(\frac{2p-1-p}{1-p}\right) S\left(p\ens[a] + \bar{p} \ens[b]\right) \\
			&= S\left(\frac{1}{2} \ens[a] + \frac{1}{2}\left(p\ens[a] + \bar{p} \ens[b]\right)\right) - \frac{1}{2} S(\ens[a]) - \frac{1}{2} S\left(p\ens[a] + \bar{p} \ens[b]\right) \\
			&= MS(\ens[a], p\ens[a] + \bar{p} \ens[b]).
		\end{aligned}
	\end{equation}
	The equality holds only if $p=0$.
\end{proof}
\end{mathSection}

The fact that the mixing entropy decreases from all directions as we get closer to an ensemble allows us to create a notion of open ball, which allows to prove the topology is at least Hausdorff.

\begin{mathSection}
\begin{defn}
	Given $\ens[a] \in \Ens$ and $r \in (0,1]$, an entropic open ball is the set of all ensembles for which the mixing entropy from $\ens[a]$ is within $r$. That is, $B_r(\ens[a]) = \{\ens \in \Ens \, | \, MS(\ens[a], \ens) < r\}$.
\end{defn}
\begin{coro}
	Every entropic open ball is an open set.
\end{coro}

\begin{proof}
	Since both the entropy and the mixing operation are continuous, the mixing entropy is also continuous. An entropic open ball is the reverse image of an open set through a continuous function and it is therefore an open set.
\end{proof}

\begin{prop}
	Ensemble spaces are Hausdorff topological spaces.
\end{prop}

\begin{proof}
	We will prove this in two ways. First, since the topology is second countable, the space is Hausdorff if and only if limits of sequences are unique. Let $\ens[a]_i$ be a sequence such that $\ens[a]_i \to \ens[a]$ and $\ens[a]_i \to \ens[b]$. We have $MS(\ens[a]_i, \ens[a]_i) = 0 \to 0$. Since $MS$ is continuous, we also have $MS(\ens[a]_i, \ens[a]_i) \to MS(\ens[a], \ens[b])$ which means $MS(\ens[a], \ens[b]) = 0$ and therefore $\ens[a] = \ens[b]$.
	
	As another way to show it, a space is Hausdorff if and only if any singleton $\{\ens\}$ is the intersection of all closed neighborhoods of $\ens$. For any $\ens$, the intersection of all closed neighborhoods will contain $\ens$, so we just need to show that no other elements can be in all closed neighborhoods. Consider an entropic closed ball centered around $\ens$ with radius $r>0$. It will contain an entropic open ball of the same radius and is therefore a closed neighborhood of $\ens$. Given an element $\ens[a] \neq \ens$, a closed ball with radius less than $MS(\ens, \ens[a])$ will not contain $\ens[a]$. Therefore, the only element contained in all entropic closed balls is $\ens$, which means the intersection of all closed neighborhoods of $\ens$ contains only $\ens$.
\end{proof}

\begin{remark}
	Intuitively, we would expect the entropic balls to be convex. This is true in both classical and quantum cases as the JSD is convex. However, the current axioms allow us to use a different entropy function in a space that is not orthogonally decomposable: the extreme points are not orthogonal. In this case, one is able to create open balls that are not convex.
	
	Whether this is physically relevant or not depends on whether there are additional physical constraints on the mixing entropy between three points, which is very likely.
\end{remark}

\begin{prop}
	Let $U \subseteq \Ens$ be an open set. Let $\ens[a] \in U$ and $L \subseteq \Ens$ be a line containing $\ens[a]$. Then there is an $r \in (0,1]$ such that for all $\ens \in L$ such that $MS(\ens[a], \ens) < r$ we have $\ens \in U$.
\end{prop}

\begin{figure}[H]
	\centering
	\begin{tikzpicture}
	%\draw [help lines] (0,0) grid (4,3);
	
	\node[above] at (2,2.6) {$\Ens$};
	
	\draw[-, thick] plot [smooth cycle] coordinates {(0.5,1.5)(1.5,0.3)(2.7,1)(2.6,2.5)(1,2.5)};
	
	\draw[-, thin] plot [smooth cycle] coordinates {(1.0,1.5)(1.75,1.0)(2.4,1.5)(2.3,2.2)(1.8,1.8)};
	\node[below right] at (2.05,1.35) {$U$};

	\coordinate (A) at (0,0.7);
	\coordinate (B) at (3.5,2.5);

	\def\a{0.4}
	\def\b{0.5}
	\def\c{0.6}

	\filldraw [black] ($(A)!\b!(B)$) circle (1pt) node[below] {$\ens[a]$};
	\draw [dotted,thick,gray] ($(A)!\b!(B)$) circle (12pt);
	
	\draw[dashed,gray] (A) -- ($(A)!\a!(B)$);
	\draw[-, thick] ($(A)!\a!(B)$) -- ($(A)!\b!(B)$);
	\draw[-, thick] ($(A)!\b!(B)$) -- ($(A)!\c!(B)$);
	\draw[dashed,gray] ($(A)!\c!(B)$) -- (B) node[below,black] {$L$};
	%\draw[dashed] (2, 2) -- (3, 2.5);
	%\draw[-, thick] (1.5, 1.75) -- (2, 2) node[below] {$b$};
	%\draw[dashed] (0.4,1.2) -- (1.5, 1.75) node[below] {$a$};
	\end{tikzpicture}
\end{figure}

\begin{proof}
	Let $U \subseteq \Ens$ be an open set, $\ens[a] \in U$ and $L \subseteq \Ens$ be a line containing $\ens[a]$. Take $\ens[b] \in L$ such that $\ens[b] \neq \ens[a]$. Let $f : [0,1] \to \Ens$ be such that $f(p) \mapsto p \ens[a] + \bar{p} \ens[b]$. Since the mixing operation is continuous, then $f$ is also continuous. This means that $f^{-1}(U)$ is an open subset of $[0,1]$. Since $\ens[a] \in U$, $1 \in f^{-1}(U)$ and therefore there will be an open interval $(\lambda, 1]$ such that $(\lambda, 1] \subseteq f^{-1}(U)$. Let $\ens[c] = \lambda \ens[a] + \bar{\lambda} \ens[b]$ and $r = MS(\ens[a], \ens[c])$. By \ref{pm-es-mixingEntropyMonotonic}, we have that, for all $p \in (0,1]$, $MS(\ens[a], p\ens[a] +\bar{p}\ens[c])<r$. We also have $p\ens[a] +\bar{p}\ens[c] = p\ens[a] +\bar{p}(\lambda \ens[a] + \bar{\lambda} \ens[b]) = (p + \bar{p}\lambda)\ens[a] + \bar{p}\bar{\lambda}\ens[b]$ where $p + \bar{p}\lambda \in (\lambda, 1]$. Therefore $p\ens[a] +\bar{p}\ens[c] \in f((\lambda,1]) \subseteq U$.
	
	If $\ens[a]$ is an extreme point of $L$ (i.e. there is no $\ens[d] \in L$ such that $p\ens[b] + \bar{p}\ens[d] = \ens[a]$), then we are done. If $\ens[a]$ is not an extreme point, then we can find $\ens[d] \in L$ such that $p\ens[b] + \bar{p}\ens[d] = \ens[a]$. Repeat the previous procedure with $\ens[d]$ to find another $r'$ and let $\hat{r}$ be the minimum between the two. Then we have that $\hat{r} \in (0,1]$ such that $\ens \in U$ for all $\ens \in L$ such that $MS(\ens[a], \ens) < \hat{r}$.
\end{proof}
\end{mathSection}
	
There are a number of conjectures on the topology generated by the entropic open ball that are probably all related. They are summarized here with some notes on what has been understood so far.

\begin{conj}
	For every sequence $\ens[a]_i \in \Ens$, $\ens[a]_i \to \ens[a]$ if and only if $MS(\ens[a], \ens[a]_i) \to 0$.
\end{conj}

\begin{conj}
	The topology of the ensemble space is generated by the entropic open balls.
\end{conj}

\begin{conj}
	Let $U \subseteq \Ens$ be an open set and $\ens[a] \in U$. Then there exists an $r \in (0,1]$ such that $B_r(\ens[a]) \subseteq U$.
\end{conj}

\begin{remark}
	These conjectures are likely the same. If the topology is generated by the entropic open balls, then convergence in mixing entropy is the convergence criterion associated with the topology. It also means that we can generate every open set with entropic open balls by fitting them inside every open set.
	
	Note that all entropic open balls are open sets in the topology, since the mixing entropy is a continuous function. It suffices to prove that all open sets can be generated by open balls. We have shown that given a point in an open set we can find a finite range of mixing entropy along each direction. However, since we do not know if there is a non-zero infimum across all directions, we do not know whether we can ``fit'' an entropic open ball.
	
	Similarly, it is clear that if a sequence converges, the mixing entropy goes to zero. However, we do not know whether there is an open set for which the mixing entropy goes to zero but the sequence does not converge. This would mean that a sequence always eventually enters an entropic open ball of any size, but it may not enter all open sets.
\end{remark}

\begin{conj}
	Let $\ens[a], \ens[b]_i\in \Ens$ and $\ens[c]_i = p \ens[a] +\bar{p} \ens[b]_i$ for some $p \in (0,1)$. Then $\ens[b]_i \to \ens[b]$ if and only if $\ens[c]_i \to \ens[c]$. 
\end{conj}

\begin{conj}[Affine combinations are continuous]
	Let $\ens \in \Ens$ and let $p \in [0,1]$. Let $+_{p\ens[a]} : \Ens \to \Ens$ be the curried function $+_{p\ens[a]}(\ens[b]) = p\ens[a] + \bar{p}\ens[b]$. The function is a homeomorphism between $\Ens$ and $+_{p\ens[a]}(\Ens)$.
\end{conj}

\begin{conj}
	Mixing is an open map.
\end{conj}

\begin{conj}
	An ensemble space embeds continuously into a topological vector space.
\end{conj}

\begin{conj}
	Let $+_{\ens[a]\ens[b]} : [0,1] \to \Ens$ be the curried function $+_{\ens[a]\ens[b]}(p) = p\ens[a] + \bar{p}\ens[b]$. The function is a homeomorphism between $[0,1]$ and $+_{\ens[a]\ens[b]}([0,1])$.
\end{conj}

\begin{remark}
	These are all about showing that mixing allows a continuous inverse. It is easy to show that mixing gives us a continuous bijection. The problem is showing that the inverse is continuous.
	
	Continuity of mixtures only shows that we can stretch open sets to bigger open sets. What we would need is to show that we can shrink open sets to open sets. For example, suppose $U$ is an open neighborhood of $\ens[a]$. The set $V = p \ens[a] + \bar{p} U$ is a set that contains $\ens[a]$ and it is a subset of $U$. The question is whether a sequence can stay in $U$ and converge to $\ens[a]$ without ever entering $V$.
\end{remark}

Another open question is whether the mixing entropy, or the entropy directly, can be related to a generalized notion of inner product.

\subsection{Entropic metric}

Following the first observation that the mixing entropy is a pseudo-distance, the second observation is that the strict concavity of the entropy forces the Hessian to be negative definite. The negation of the Hessian, then, is a positive definite function of two variations. In general, the Hessian of a scalar function is not a tensor. However, the ensemble space is a linear space and that linearity has physical significance. It is only for coordinates linear with respect to the mixing coefficient that the convex combination of coordinates will equal the statistical mixture of the corresponding ensembles. Therefore we can define a metric tensor which corresponds to the negative Hessian calculated on that linear structure. When using non-linear coordinates, as long as the coordinates are smooth, one can always make the appropriate transformation to linear coordinates.

\begin{mathSection}
\begin{remark}
	In this section we will assume that an ensemble space embeds continuously into a topological vector space, even though we have not yet proved it. We are going to talk about variations $\delta \ens$ on the space, even though it is not yet clear exactly which mathematical approach is best to make variations well-defined. We are also going to talk about a metric tensor even though the space is not, in general, a manifold. All these issues will need to be resolved in due time.
\end{remark}
	
\begin{defn}
	An ensemble space is \textbf{smooth} if the entropy is twice differentiable with respect to the mixing coefficients.
\end{defn}

\begin{prop}
	Discrete and continuous classical ensemble spaces and quantum ensemble spaces are smooth.
\end{prop}

\begin{proof}
	In both the discrete classical case and the quantum case the entropy of an ensemble is the Shannon entropy of a decomposition in terms of pure states. The Shannon entropy is a smooth function of the coefficients. For the continuous classical case, the differential entropy is a smooth function of the probability density, which is linear under mixture. Therefore the entropy is always smooth with respect to mixing coefficients.
\end{proof}

\begin{defn}
	Assuming $\Ens$ embeds in a topological vector space and given $\ens \in \Ens$, a \textbf{variation} $\delta \ens$ of $\ens$ is a vector in the ambient space such that $\forall t \in [0,1]$ $\ens + t \delta \ens \in \Ens$. The space of all variations at $\ens$ is noted $T_{\ens}$. Unless otherwise noted, we assume the variation is expressed on the affine structure. Note that it can be re-expressed through a non-linear map as long as the map is differentiable (i.e. it maps variations to variations).
\end{defn}

\begin{remark}
	Note that since an ensemble space is a convex subset of a real vector space, coordinate systems that are linear with respect to the vector space are privileged. It is only in these coordinates, in fact, that linear combinations correspond to mixtures. The strict concavity of the entropy is therefore guaranteed in these coordinates and only these coordinates. Given the special physical significance of these coordinates, all differential objects and properties will be defined in these coordinates.
\end{remark}

\begin{defn}
	Given an ensemble $\ens \in \Ens$ and a variation $\delta \ens$ defined at that point, the \textbf{norm} of $\delta \ens$ is given by
	$$ \lVert \delta \ens \rVert_{\ens} = \sqrt{ 8 MS(\ens, \ens + \delta \ens) }.$$
	The \textbf{metric tensor} (i.e. the inner product between $\delta \ens_1 , \delta \ens_2 \in T_{\ens}$) is given by
	$$ g_{\ens}( \delta \ens_1, \delta \ens_2 ) = \frac{1}{2} \left( \lVert \delta \ens_1 + \delta \ens_2 \rVert_{\ens}^2 - \lVert \delta \ens_1 \rVert_{\ens}^2 - \lVert \delta \ens_2 \rVert_{\ens}^2 \right).$$
\end{defn}

\begin{thrm}
	Let $\Ens$ be a smooth ensemble space. Then, on the affine structure, we have
	$$ \lVert \delta \ens \rVert_{\ens}^2 = - \frac{\partial^2 S}{\partial \ens^2}(\delta \ens , \delta \ens) $$
	and 
	$$ g_{\ens}( \delta \ens_1, \delta \ens_2 ) = - \frac{\partial^2 S}{\partial \ens ^2}( \delta \ens_1, \delta \ens_2 ).$$
\end{thrm}

\begin{proof}
	To recover the first two expressions, we simply have to calculate the leading term. Since the entropy is twice differentiable, on the affine structure, we can expand it as
	\begin{equation}
		\begin{aligned}
			S(\ens + \delta \ens) &= S(\ens) + \frac{\partial S}{\partial \ens} \delta \ens + \frac{1}{2} \frac{\partial^2 S}{\partial \ens^2} \delta \ens \delta \ens + O(\delta \ens ^3).
		\end{aligned}
	\end{equation}
	Expanding the definition of $MS$, we have
	\begin{equation}
		\begin{aligned}
			MS(\ens, \ens + \delta \ens) &= S\left(\frac{1}{2} \ens + \frac{1}{2}(\ens + \delta \ens) \right) - \frac{1}{2} S(\ens) - \frac{1}{2} S(\ens+\delta \ens) \\
			&=  S\left(\ens + \frac{1}{2} \delta \ens \right) - \frac{1}{2} S(\ens) - \frac{1}{2} S(\ens+\delta \ens) \\
			&= S(\ens) + \frac{\partial S}{\partial \ens} \frac{1}{2}\delta \ens + \frac{1}{2} \frac{\partial^2 S}{\partial \ens^2} \frac{1}{2}\delta \ens \frac{1}{2}\delta \ens + O(\delta \ens ^3) \\
			&- \frac{1}{2} S(\ens) - \frac{1}{2} \left( S(\ens) + \frac{\partial S}{\partial \ens} \delta \ens + \frac{1}{2} \frac{\partial^2 S}{\partial \ens^2} \delta \ens \delta \ens + O(\delta \ens ^3) \right) \\
			&= S(\ens) + \frac{1}{2} \frac{\partial S}{\partial \ens} \delta \ens + \frac{1}{8} \frac{\partial^2 S}{\partial \ens^2} \delta \ens \delta \ens \\
			&- S(\ens) - \frac{1}{2} \frac{\partial S}{\partial \ens} \delta \ens - \frac{1}{4} \frac{\partial^2 S}{\partial \ens^2} \delta \ens \delta \ens + O(\delta \ens ^3) \\
			&= - \frac{1}{8} \frac{\partial^2 S}{\partial \ens^2} \delta \ens \delta \ens + O(\delta \ens ^3).
		\end{aligned}
	\end{equation}
	
	Therefore
	$$ \lVert \delta \ens \rVert^2 = 8 MS(\ens, \ens + \delta \ens) = -  \frac{\partial^2 S}{\partial \ens^2} (\delta \ens, \delta \ens).$$
	
	We can now substitute the norm in the definition of the metric tensor. We have
	\begin{equation}
		\begin{aligned}
			g_{\ens}(\delta \ens_1, \delta \ens_2 ) &= \frac{1}{2} \left( \lVert \delta \ens_1 + \delta \ens_2 \rVert^2 - \lVert \delta \ens_1 \rVert^2 - \lVert \delta \ens_2 \rVert^2 \right) \\
			&= \frac{1}{2} \left( - \frac{\partial^2 S}{\partial \ens^2} (\delta \ens_1 + \delta \ens_2, \delta \ens_1 + \delta \ens_2) + \frac{\partial^2 S}{\partial \ens^2} (\delta \ens_1, \delta \ens_1) + \frac{\partial^2 S}{\partial \ens^2} (\delta \ens_2, \delta \ens_2) \right) \\
			&= - \frac{1}{2} \left(  \frac{\partial^2 S}{\partial \ens^2} (\delta \ens_1, \delta \ens_1) + \frac{\partial^2 S}{\partial \ens^2} (\delta \ens_1, \delta \ens_2) + \frac{\partial^2 S}{\partial \ens^2} (\delta \ens_2, \delta \ens_1) + \frac{\partial^2 S}{\partial \ens^2} (\delta \ens_2, \delta \ens_2) \right. \\
			&-  \left. \frac{\partial^2 S}{\partial \ens^2} (\delta \ens_1, \delta \ens_1) - \frac{\partial^2 S}{\partial \ens^2} (\delta \ens_2, \delta \ens_2) \right) \\
			&= - \frac{\partial^2 S}{\partial \ens^2} (\delta \ens_1, \delta \ens_2). \\
		\end{aligned}
	\end{equation}
\end{proof}
\end{mathSection}

We now show that the metric tensor we defined reduces to the Fisher-Rao metric in the classical case and to its quantum equivalent in the quantum case.

\begin{mathSection}
\begin{prop}
	For a continuous classical ensemble space, the metric corresponds to the Fisher-Rao metric.
\end{prop}

\begin{proof}
	Let $\Ens$ be a continuous classical ensemble space. Each ensemble is a classical probability density $\rho$. Let $V \subseteq \Ens$ be a manifold of probability distributions over $X$ parametrized by $\theta^i$. The \href{https://en.wikipedia.org/wiki/Fisher_information_metric}{Fisher-Rao metric} is defined as:
	\begin{equation}
		g_{ij} =- \int_X \frac{\partial^2 \log \rho}{\partial \theta^i \partial \theta^j} \rho dx.
	\end{equation}
	where $\log$ will be the natural logarithm throughout this calculation. Recall that for a continuous classical ensemble, the entropy is given by
	\begin{equation}
		S(\rho) =- \int_X \rho \log \rho dx.
	\end{equation}
	
	Let us now calculate the first two terms in the Taylor expansion around $\rho$ with a variation $\delta\rho$. Recall that
	\begin{equation}
		\begin{aligned}
			\log(x+dx) &= \log(x) + d_x \log x \, dx + \frac{1}{2} d_x d_x  \log x \, dx^2 + O(dx^3) \\
			&= \log(x) + \frac{1}{x} dx - \frac{1}{2} \frac{1}{x^2} dx^2 + O(dx^3).
		\end{aligned}
	\end{equation}
	We have
	\begin{equation}
		\begin{aligned}
			S(\rho + \delta \rho) &= -\int_X (\rho + \delta \rho) \log (\rho + \delta \rho) dx \\
			&=-\int_X (\rho + \delta \rho) \left[\log \rho + \frac{1}{\rho} \delta \rho - \frac{1}{2\rho^2} \delta \rho^2 + O(\delta \rho^3)\right] dx \\
			&= - \int_X \rho \log \rho dx - \int_X \left[\log \rho + 1 \right]dx \delta \rho -\int_X \left[\frac{1}{\rho} - \frac{\rho}{2\rho^2}\right] dx \delta \rho^2 + \int_X dx O(\delta\rho^3) \\
			&= - \int_X \rho \log \rho dx - \int_X \left[\log \rho + 1 \right]dx \delta \rho - 	
			\frac{1}{2}\int_X \frac{1}{\rho} dx \delta \rho^2 + \int_X dx O(\delta\rho^3) \\
		\end{aligned}
	\end{equation}
	\begin{equation}
		\begin{aligned}
			\frac{\partial^2 S}{\partial \rho^2}(\delta \rho, \delta \rho) &= -\int_X \frac{1}{\rho} \delta \rho^2 dx = -\int_X \frac{1}{\rho} \delta \rho^2 dx + 0 = -\int_X \frac{1}{\rho} \delta \rho^2 dx + \delta^2 (1) \\
			&= -\int_X \frac{1}{\rho} \delta \rho^2 dx + \delta^2 \int_X \rho dx = -\int_X \frac{1}{\rho} \delta \rho^2 dx + \int_X \delta^2 \rho dx \\
			&= \int_X \rho dx \left[-\frac{1}{\rho^2} \delta \rho^2 + \frac{1}{\rho} \delta^2 \rho \right]
			= \int_X \rho dx \, \delta \left[\frac{1}{\rho} \delta \rho \right] \\
			&= \int_X \rho dx \, \delta^2 \log \rho.
		\end{aligned}
	\end{equation}
	
	Let us consider a family of ensembles charted by a set of parameters $\theta^i$, not necessarily forming a linear chart. We have
	\begin{equation}
		\begin{aligned}
			g_{\ens}(d\theta^i, d\theta^j) &= - \frac{\partial^2 S}{\partial \rho^2}\left(\frac{\partial 
				\rho}{\partial \theta^i} d\theta^i, \frac{\partial 
				\rho}{\partial \theta^j} d\theta^j\right) = - \int_X \rho dx \frac{\partial^2 \log \rho}{\partial \theta^i \partial \theta^j}d\theta^i d\theta^j
		\end{aligned}
	\end{equation}
	which recovers the Fisher-Rao metric.
	
	This same calculation can be done in the case of a discrete classical space.
\end{proof}

\begin{remark}
	For quantum mechanics, the situation is more complicated as there are different definitions of Fisher metrics\footnote{For details, see \href{https://iopscience.iop.org/article/10.1088/1367-2630/ac1186}{Vishal Katariya and Mark M Wilde 2021 New J. Phys. 23 073040} or \href{https://link.springer.com/chapter/10.1007/978-4-431-54493-7_4}{Watanabe, Y. (2014). Quantum Estimation Theory}.}. Additionally, we will take steps in the calculations that are already present in established literature, though the mathematical conditions for those steps to be well-defined are unclear. We will therefore just show a general connection, without worrying about the mathematical details.
	
\end{remark}

\begin{prop}
	For a quantum ensemble space, the metric corresponds to the Bures metric and the quantum Fisher information metric.
\end{prop}

\begin{proof}
	Since we are working in a quantum ensemble space, each ensemble is a density operator $\rho$. The entropy is given by the von Neumann entropy
	\begin{equation}
		S(\rho) = - \tr\left(\rho \log \rho\right).
	\end{equation}
	where $\log$ will be the natural logarithm throughout this calculation. We take the first variation and have
	\begin{equation}
		\begin{aligned}
			\delta S(\rho) &= - \delta \tr\left(\rho \log \rho\right)= - \tr\left(\delta \rho \log \rho + \rho \delta \log \rho\right) \\
			&= - \tr\left(\delta \rho \log \rho + \rho \rho^{-1} \delta \rho \right) \\
			&= - \tr\left(\left(\log \rho + 1 \right) \delta \rho \right).
		\end{aligned}
	\end{equation}
	We take the second variation and have
	\begin{equation}
		\begin{aligned}
			\delta^2 S(\rho) &= - \delta \tr\left(\left(\log \rho + 1 \right) \delta \rho \right) \\
			&= - \tr\left(\delta \left(\log \rho + 1 \right) \delta \rho + \left(\log \rho + 1 \right) \delta \delta \rho \right) \\
			&= - \tr\left(\rho^{-1} \delta \rho \delta \rho + \left(\log \rho + 1 \right) \delta \delta \rho \right).
		\end{aligned}
	\end{equation}
	Note that $\delta \rho$ is defined in a linear chart, therefore $\delta \delta \rho = 0$. Therefore we have 
	\begin{equation}
		\begin{aligned}
			\frac{\partial^2 S}{\partial \rho^2}(\delta \rho, \delta \rho) &= - \tr\left(\rho^{-1} \delta \rho \delta \rho \right).
		\end{aligned}
	\end{equation}
	Let us consider a family of ensembles charted by a set of parameters $\theta^i$, not necessarily forming a linear chart. We have
	\begin{equation}
		\begin{aligned}
			g_{\ens}(d\theta^i, d\theta^j) &= - \frac{\partial^2 S}{\partial \rho^2}\left(\frac{\partial 
				\rho}{\partial \theta^i} d\theta^i, \frac{\partial 
				\rho}{\partial \theta^j} d\theta^j\right) = \tr\left( \rho^{-1} \frac{\partial \rho}{\partial \theta^i }\frac{\partial \rho}{\partial \theta^j}\right) d\theta^i d\theta^j
		\end{aligned}
	\end{equation}
	which recovers one version of the quantum Fisher-Rao metric.
	
	Noting that the right logarithmic derivative (RLD) $L^R$ satisfies $\frac{\partial \rho}{\partial \theta} = \rho L^R$, we can write 
	\begin{equation}
		\begin{aligned}
			g_{\ens}(d\theta^i, d\theta^j) &= \tr\left( \rho^{-1} \frac{\partial \rho}{\partial \theta^i }\frac{\partial \rho}{\partial \theta^j}\right) d\theta^i d\theta^j = \tr(\rho^{-1}\rho L^R
			_i \rho L^R_j)d\theta^i d\theta^j = \tr(\rho L^R
			_i L^R_j)d\theta^i d\theta^j
		\end{aligned}
	\end{equation}
	which recovers the RLD Fisher information.
\end{proof}

\begin{remark}
	The feedback we received on the above derivation is contradictory, as some say we are handling the possible non-commutativity between $\rho$ and its variation $\delta \rho$ correctly and others say we aren't. Note that in all steps we are using the cyclical nature of the trace, and not commuting $\rho$ and its variation $\delta \rho$.
\end{remark}

\begin{remark}
	Note that these results relate also to the \href{https://en.wikipedia.org/wiki/Ruppeiner_geometry}{Ruppeiner metric}, which is the negation of the Hessian of the entropy. They also relate to the Hessian metric as described in \href{https://link.springer.com/chapter/10.1007/978-3-642-40020-9_4}{Shima, H. (2013). Lecture Notes in Computer Science, vol 8085}.
\end{remark}
\end{mathSection}

\section{State capacity}

In this section we are going to develop a generalized version of the count of states. In classical statistical mechanics, the entropy for a uniform distribution $\rho_U$ over a region of phase space $U$ is given by $S(\rho_U) = \log \mu(U)$, where $\mu$ is the count of states given by the Liouville measure. Since entropy is our starting point, we will define the state capacity of a set of ensembles as the highest exponential of the entropy achievable by mixing those ensembles. This definition works in general, and recovers both the classical Liouville measure and the dimensionality of a quantum subspace.

Given an ensemble, we want to quantify the number of distinguishable cases over which the ensemble is spread. Since entropy characterizes the variability of the preparations of the ensemble, greater variability means the ensemble is spread over more cases. Therefore the count of distinguishable states must be a monotonic function of the entropy. The question is what function.

The first hint that the exponential is the right function is the relationship $S(\rho_U) = \log \mu(U)$ in classical statistical mechanics. Another hint is that we will want this size to be multiplicative over independent distributions. That is, if we have two ensembles $\rho_1 \in \Ens_1$ and $\rho_2 \in \Ens_2$ of two different ensemble spaces, we can imagine the composite system $\rho = \rho_1 \rho_2$ representing the ensemble of the two independent ensembles. While the entropy should be additive, that is $S(\rho) = S_1(\rho_1) + S_2(\rho_2)$, the count of states should be multiplicative, that is $\mu(\rho) = \mu_1(\rho_1) \mu_2(\rho_2)$. This suggests the relationship $S(\rho) = \log \mu (\rho)$ for all ensembles. The other hint is that the spread can be, at most, additive. If we mix two ensembles, the spread can be at most over the configurations of both and not more. The following shows that the exponential of the entropy has exactly that property.\footnote{
	Note that the state capacity for a single element corresponds to the ``ensemble volume'' defined in \href{https://journals.aps.org/pra/abstract/10.1103/PhysRevA.59.2602}{M.J.W. Hall 1999, Phys. Rev. A 59, 2602} and more recently \href{https://iopscience.iop.org/article/10.1088/1751-8121/aad50f}{M.J.W. Hall 2018 J. Phys. A: Math. Theor. 51 364001}. It is connected to various uncertainty relationships, including a stronger form of the quantum uncertainty principle. It would be interesting to understand what results can be generalized to the ensemble space.}

\begin{mathSection}
\begin{prop}[Exponential entropy subadditivity]\label{pm-es-exponentialEntropySubadditivity}
	Let $\ens[a], \ens[b] \in \Ens$ and $\ens = p \ens[a] + \bar{p} \ens[b]$ for some $p \in [0,1]$. Then $2^{S(\ens)} \leq 2^{S(\ens[a])} + 2^{S(\ens[b])}$, with the equality holding if and only if $\ens[a] \ortho \ens[b]$ and $p = \frac{2^{S(\ens[a])}}{2^{S(\ens[a])} + 2^{S(\ens[b])}}$.
\end{prop}

\begin{proof}
	If $p$ is fixed, the upper variability bound of entropy is saturated only if $\ens[a]$ and $\ens[b]$ are orthogonal by definition. Therefore the entropy maximum for the mixed ensemble will be achieved when the elements are orthogonal, for some value of $p$.
	
	Now fix the entropy of $\ens[a]$ and $\ens[b]$ to some values $S_a = S(\ens[a])$ and $S_b = S(\ens[b])$. The entropy of the mixture depends only on $p$, so we need to find the $p$ that maximizes the expression. Since $\ens[a]$ and $\ens[b]$ are orthogonal, $S(p \ens[a] + \bar{p}\ens[b]) =  - p \log p - \bar{p} \log \bar{p} + p S_a + \bar{p} S_b$ which is a smooth function of $p$. Let us find the maximum, which is a stationary point.
	\begin{equation}
		\begin{aligned}
			0 &= \frac{d}{dp} S(p \ens[a] + \bar{p}\ens[b]) =\frac{d}{dp} \left( - p \log p - \bar{p} \log \bar{p} + p S_a + \bar{p} S_b \right) \\
			&= - \log p - 1 + \log \bar{p} + 1 + S_a - S_b \\
			\log \frac{p}{\bar{p}} &= \log 2^{S_a} - \log 2^{S_b} \\
			\log \frac{p}{1-p} &= \log \frac{2^{S_a}}{2^{S_b}}  \\
			p 2^{S_b} &= (1-p) 2^{S_a}  \\
			p (2^{S_a} + 2^{S_b}) &= 2^{S_a}  \\
			p &= \frac{2^{S_a}}{2^{S_a} + 2^{S_b}}  \\
		\end{aligned}
	\end{equation}
	Note that since the entropy is strictly concave, the stationary point must correspond to a maximum. Having found the value of $p$ that maximizes the entropy, we can calculate the maximum entropy.
	\begin{equation}
		\begin{aligned}
			\bar{p} &= 1- \frac{2^{S_a}}{2^{S_a} + 2^{S_b}} = \frac{2^{S_b}}{2^{S_a} + 2^{S_b}} \\
			S(p \ens[a] + \bar{p}\ens[b]) &= - p \log p - \bar{p} \log \bar{p} + p S_a + \bar{p} S_b  \\
			&= - \frac{2^{S_a}}{2^{S_a} + 2^{S_b}} \log \frac{2^{S_a}}{2^{S_a} + 2^{S_b}} - \frac{2^{S_b}}{2^{S_a} + 2^{S_b}} \log \frac{2^{S_b}}{2^{S_a} + 2^{S_b}} \\
			&+ \frac{2^{S_a}}{2^{S_a} + 2^{S_b}} \log 2^{S_a} + \frac{2^{S_b}}{2^{S_a} + 2^{S_b}} \log 2^{S_b} \\
			&= \frac{2^{S_a}}{2^{S_a} + 2^{S_b}} \log \left( 2^{S_a} + 2^{S_b} \right) + \frac{2^{S_b}}{2^{S_a} + 2^{S_b}} \log \left( 2^{S_a} + 2^{S_b} \right) \\
			&= \frac{2^{S_a} + 2^{S_b}}{2^{S_a} + 2^{S_b}} \log \left( 2^{S_a} + 2^{S_b} \right) \\
			\log 2^{S(p \ens[a] + \bar{p}\ens[b])} &= \log \left( 2^{S_a} + 2^{S_b} \right) \\
			2^{S(p \ens[a] + \bar{p}\ens[b])} &=  2^{S_a} + 2^{S_b}  \\
		\end{aligned}
	\end{equation}
	Therefore the maximum entropy obtainable through a mixture is $S(p \ens[a] + \bar{p}\ens[b]) = \log (2^{S(\ens[a])} + 2^{S(\ens[b])})$ which is obtained when $\ens[a]$ and $\ens[b]$ are orthogonal and $p = \frac{2^{S(\ens[a])}}{2^{S(\ens[a])} + 2^{S(\ens[b])}}$.
\end{proof}

\begin{remark}
	We have proved that the exponential of the entropy gives us subadditivity. Ideally, we should prove that this is the only function of the entropy with that characteristic.
\end{remark}

\begin{conj}
	Let $f : \mathbb{R} \to \mathbb{R}$ be a continuous function such that $f(S(p\ens[a]+\bar{p}\ens[b])) \leq f(S(\ens[a])) + f(S(\ens[b]))$ and the equality can be verified in some condition. Then $f(x) = \kappa^x$ for some arbitrary constant $\kappa$.
\end{conj}
\end{mathSection}

Given a set of ensembles, we can ask what spread is reachable by a mixture in terms of the number of distinguishable states. We call this the state capacity as it represents the maximum potential spread reachable by the set, and because it turns out to be a non-additive measure.\footnote{In some literature, capacities are non-additive measures.} The state capacity is monotone, subadditive and recovers additivity over orthogonal sets.

\begin{mathSection}
\begin{defn}
	Let $A \subseteq \Ens$ be a subset of an ensemble space. The \textbf{state capacity} of $A$ is defined as $\scap(A) = \sup(2^{S(\hull(A))}\cup\{0\})$.
\end{defn}

\begin{coro}
	The state capacity is a convex supremum of the exponential of the entropy with $0$ for the empty set.
\end{coro}

\begin{coro}
	Given a set $A \subseteq \Ens$, the state capacity of all hulls is the same. That is, $\scap(\hull(A)) = \scap(\shull(A)) = \scap(\chull(A))$.
\end{coro}

\begin{proof}
	Since the entropy is continuous, by \ref{pm-es-convexSupremumContinuous} $\scap(A) = \scap(\hull(A)) = \scap(\shull(A)) = \scap(\chull(A))$.
\end{proof}

\begin{prop}
	The state capacity is a set function that is
	\begin{enumerate}
		\item non-negative: $\scap(A) \in [0, +\infty]$
		\item increasing: $A \subseteq B \implies \scap(A) \leq \scap(B)$
		\item subadditive: $\scap(A \cup B) \leq \scap(A) + \scap(B)$
		\item additive over orthogonal sets: $A \ortho B \implies \scap(A \cup B) = \scap(A) + \scap(B)$ 
		\item continuous from below.
	\end{enumerate}
\end{prop}

\begin{proof}
	1. By \ref{pm-es-convexSupremumProps}, $\scap(A) \in [0, \sup(2^{S(\Ens)})] \subseteq [0,+\infty]$. 
	
	2. By \ref{pm-es-convexSupremumProps}.
	
	3. Let $A, B \subseteq \Ens$ and let $\ens = p \ens[a] + \bar{p} \ens[b]$ for some $p \in [0,1]$, $\ens[a] \in A$ and $\ens[b] \in B$. By \ref{pm-es-exponentialEntropySubadditivity} and the definition of state capacity, $2^{S(\ens)} \leq 2^{S(\ens[a])} + 2^{S(\ens[b])} \leq \scap(A) + \scap(B)$. Since this is true for any finite mixture of elements of $A$ and $B$, it will be true for any element of $\hull(A \cup B)$. Consequently, the supremum of the exponential entropy cannot exceed the sum of the state capacities. Therefore $\scap(A \cup B) \leq \scap(A) + \scap(B)$ (i.e. the state capacity is subadditive).
	
	4. Let $A, B \subseteq \Ens$ be two orthogonal subsets. Let $\{\ens[a]_i\} \subseteq A$ be a sequence of ensembles such that $2^{S(\ens[a]_i)} \to \scap(A)$ and let $\{\ens[b]_i\} \subseteq B$ be a sequence of ensembles such that $2^{S(\ens[b]_i)} \to \scap(B)$. Consider $\ens_i = p_i \ens[a]_i + \bar{p_i} \ens[b]_i$ where $p_i = \frac{2^{S(\ens[a]_i)}}{2^{S(\ens[a]_i)} + 2^{S(\ens[b]_i)}}$. Then, by \ref{pm-es-exponentialEntropySubadditivity}, $2^{S(\ens_i)} = 2^{S(\ens[a]_i)} + 2^{S(\ens[b]_i)}$. This means that $2^{S(\ens_i)} \to \scap(A) + \scap(B)$. Therefore $\scap(A \cup B)$ must be at least $\scap(A) + \scap(B)$. Combining with the previous result, $\scap(A \cup B) = \scap(A) + \scap(B)$. Therefore the state capacity, as a set function, is additive over orthogonal sets of ensembles.

	5. By \ref{pm-es-convexSupremumProps}.
\end{proof}

\begin{remark}
	The state capacity is not continuous from above. Let $\Ens$ be an ensemble space and $\ens[a], \ens[b] \in \Ens$ two distinct ensembles. Without loss of generality, let $S(\ens[a]) \leq S(\ens[b])$. Then $S(p \ens[a] + \bar{p} \ens[b]) \geq S(\ens[a])$ for all $p \in [0,1]$. Let $\{ \ens_i \} \subset \hull(\{\ens[a], \ens[b]\})$ be a sequence of mixtures of $\ens[a]$ and $\ens[b]$. Let $A_j = \{ \ens_i \, | \, i \geq j\}$ be the family of sets that contains all the elements of the sequence starting from $j$ respectively. This is a decreasing sequence, since $A_j \supseteq A_{j+i}$. We have that $\scap(A_j) \geq 2^{S(\ens[a])} > 0$ for all $j$, which means that $\lim\limits_{i \to \infty} \scap(A_i) \geq 2^{S(\ens[a])} > 0$. However, $\bigcap A_j = \emptyset$ which means $\scap(\lim\limits_{i \to \infty} A_i) = 0$. This means that, in general, $\scap(\lim\limits_{i \to \infty} A_i) \neq \lim\limits_{i \to \infty} \scap(A_i)$ for a decreasing sequence.
\end{remark}

\end{mathSection}

The state capacity is additive over orthogonal sets. Intuitively, orthogonal sets correspond to mutually exclusive events, so it makes sense that the count of states is additive. We would expect the converse to be true as well: if the count of states is additive, the sets correspond to mutually exclusive events and therefore the sets are orthogonal. This leads to the following conjecture:

\begin{conj}
	The state capacity is additive only on orthogonal sets. That is, if $\scap(A)<\infty$, $\scap(B)<\infty$ and $\scap(A \cup B) = \scap(A) +\scap(B)$, then $A \ortho B$.
\end{conj}

One should be able to prove that if the state capacity adds, then the states with highest entropy satisfy the upper variability bound and are orthogonal. What needs to be shown is that this is enough to say all elements are orthogonal. There is a series of nice results connected to this to expand the whole theory. If $\ens[a] \ortho \ens[b]$ then $\ens[a]$ is orthogonal to all the components of $\ens[b]$. If $\ens[a]$ is the ensemble with maximum entropy in $\chull(A)$, it should be true that all elements of the hull are components of $\ens[a]$. One should be able to show that, given two sequences with a convergent entropy and whose elements are orthogonal to each other, one can create another sequence that converges to an entropy higher than both of them. Another useful tool would be a type of sequence within $\hull(A)$ whose entropy always increases, tends to the supremum and each element is a component of the following. Also, the state with maximal entropy (if it exists) is an internal point. All internal points are in the $\sigma$-hull. We leave these series of conjectures for future work.

We now show that the state capacity recovers a notion of number of cases in all three target spaces. In the discrete classical case it recovers the number of points; in the continuous classical case, it recovers the Liouville volume; in the quantum case, it recovers the number of orthogonal pure states.

\begin{mathSection}
\begin{prop}
	Let $\Ens$ be a discrete classical ensemble space. Let $U \subseteq \{s_i\}$ be a subset of the corresponding (extreme) points and $A$ be the set of probability distributions whose support is a subset of $U$. Then $\scap(A) = \#U$.
\end{prop}

\begin{proof}
	Let $A$ be the set of probability distributions defined over a discrete countable set $U$. The highest entropy is achieved by the uniform distribution, which equals $\log (\#U)$. Therefore $\scap(A) = 2^{\log(\#U)} = \#U$.
\end{proof}

\begin{prop}
	Let $\Ens$ be a continuous classical ensemble space. Let $U \subseteq X$ be a subset of the corresponding symplectic manifold (i.e. phase space) and $A$ be the set of probability distributions whose support is a subset of $U$. Then $\scap(A) = \mu(U)$ where $\mu(U)$ is the Liouville measure.
\end{prop}

\begin{proof}
	Let $A$ be the set of probability distributions whose support is a subset of $U$. The highest entropy is achieved by the uniform distribution, which equals $\log(\mu(U))$. Therefore $\scap(A) = 2^{\log(\mu(U))} = \mu(U)$.
\end{proof}

\begin{prop}
	Let $\Ens$ be a quantum ensemble space. Let $U \subseteq \mathcal{H}$ be a subspace of the corresponding Hilbert space and $A$ be the set of density operators defined on that subspace (i.e. zero eigenvalues outside). Then $\scap(A) = \dim(U)$ where $\dim(U)$ is the dimensionality of the subspace.
\end{prop}

\begin{proof}
	Let $A$ be the set of density operators defined on a finite-dimensional subspace $U$. The highest entropy is achieved by the maximally mixed state, and equals $\log(\dim(U))$. Therefore $\scap(A) = 2^{\log(\dim(U))} = \dim(U)$. Now let $U$ be infinite dimensional. Then there is no upper bound on the entropy, and therefore $\scap(A) = \infty = \dim(U)$.
\end{proof}
\end{mathSection}

\section{Fraction capacity}

In this section we are going to define the \textbf{fraction capacity}, which can be understood as a generalization of probability that works with any ensemble space. In classical probability, additivity is justified by the mutual exclusivity of all elements of the sample space. In quantum mechanics, classical probability can be recovered only on orthogonal subspaces precisely because they represent mutually exclusive events. Therefore, the fraction capacity achieves the generalization not by focusing on probability of outcomes, but by focusing on mixing coefficient.


The typical way to understand probability is through outcomes of a process: $50\%$ probability for tails means that if we repeated the coin toss multiple times, we would expect roughly half to be tails. In classical mechanics, it can also be understood as the probability of preparation: roughly half the times we selected a preparation procedure that prepared tails. In quantum mechanics, this does not work as the probability used during the mixing is the same as the probability of the outcome only if we are mixing orthogonal states. Effectively, probability in the usual sense is defined only on outcomes. The fraction capacity, instead, defines a measure on preparations.

The fraction capacity tells us how much of an ensemble $\ens$ can be constructed through a mixture of ensembles from a set $A$. It is a non-negative, unit bounded, subadditive, monotone continuous (from below) measure, and it reduces to the probability measure in the classical case, and over measurement contexts in the quantum case. The goal is to create a measure theoretic generalization of probability theory that can work on all ensemble spaces.

\begin{figure}[h]
	\centering
	\def\xstep{0.75em}
	\def\bHeight{0.6}
	\def\tHeight{0.1}
	\def\tGap{0.3}
	\begin{tikzpicture}
		%\draw [help lines] (0,0) grid (10,3);
		
		\node at (1,0.5) {$\ens_{123456}$};
		
		\draw (0, 0) foreach \x in {1,...,6} { -- (\tGap * \x, 0) -- (\tGap * \x, -\tHeight) (\tGap * \x, 0) } -- ++(\tGap, 0);
		\draw plot[ybar, bar width={\tGap cm / 2}] coordinates{(\tGap * 1, \bHeight/6) (\tGap * 2, \bHeight/6) (\tGap * 3, \bHeight/6) (\tGap * 4, \bHeight/6) (\tGap * 5, \bHeight/6) (\tGap * 6, \bHeight/6)};
		
		\node at (2.6,0) {$ = \frac{1}{6}$};
		
		\node at (4.4,0.5) {$\ens_{3}$};
		
		\draw (3, 0) foreach \x in {1,...,6} { -- (3 + \tGap * \x, 0) -- (3 + \tGap * \x, -\tHeight) (3 + \tGap * \x, 0) } -- ++(\tGap, 0);
		\draw plot[ybar, bar width={\tGap cm / 2}] coordinates{(3 + \tGap * 3, \bHeight)};

		\node at (5.6,0) {$ + \frac{5}{6}$};
		
		\node at (7,0.5) {$\ens_{12456}$};
		
		\draw (6, 0) foreach \x in {1,...,6} { -- (6 + \tGap * \x, 0) -- (6 + \tGap * \x, -\tHeight) (6 + \tGap * \x, 0) } -- ++(\tGap, 0);
		\draw plot[ybar, bar width={\tGap cm / 2}] coordinates{(6 + \tGap * 1, \bHeight/5) (6 + \tGap * 2, \bHeight/5) (6 + \tGap * 4, \bHeight/5) (6 + \tGap * 5, \bHeight/5) (6 + \tGap * 6, \bHeight/5)};
	\end{tikzpicture}
	\caption{Visual representation of the fraction.}\label{pm-es-fig-fraction}
	%\includegraphics[width=0.6\textwidth]{tempimages/Fraction.jpg}
\end{figure}

First we define the \textbf{fraction} of one element with respect to another. For example, as shown in Fig. \ref{pm-es-fig-fraction}, suppose $\ens_{123456}$ is a uniform distribution for a six-faced die. Suppose $\ens_3$ represents the outcome 3 with $100\%$ probability. Then $\ens_{123456}$ can be understood as $\ens_{123456} = \frac{1}{6} \ens_3 + \frac{5}{6} \ens_{12456}$ where $\ens_{12456}$ is the uniform distribution over the outcomes 1,2,4,5 and 6. Note that $\frac{1}{6}$ is the highest coefficient we can put in front of $\ens_3$ in a convex combination and have $\ens_{123456}$ as a result, which coincides with the probability of obtaining 3 from a uniform distribution over six outcomes. That is how we define the fraction of $\ens_3$ with respect to $\ens_{123456}$.

\begin{mathSection}
	\begin{defn}
		Let $\ens, \ens[a] \in \Ens$ be two ensembles. The \textbf{fraction} of $\ens[a]$ with respect to $\ens$ is the greatest mixing coefficient for which $\ens$ can be expressed as a mixture of $\ens[a]$. That is, $\fraction_{\ens}(\ens[a]) = \sup(\{ p \in [0,1] \, | \, \exists \, \ens[b] \in \Ens \text{ s.t. }  \ens = p \ens[a] + \bar{p} \ens[b] \})$.
	\end{defn}

\begin{figure}[H]
	\centering
	\begin{tikzpicture}
		%\draw [help lines] (0,0) grid (4,3);
		
		\coordinate (A) at (3,0);
		\coordinate (B) at (0,0);
		
		\coordinate (e) at ($(A)!0.7!(B)$);
		\coordinate (a) at ($(A)!0.8!(B)$);
		\coordinate (b1) at ($(A)!0.4!(B)$);
		\coordinate (b2) at ($(A)!0.2!(B)$);
		\coordinate (b3) at ($(A)!0.1!(B)$);
		\coordinate (b4) at ($(A)!0.05!(B)$);
		\coordinate (b5) at ($(A)!0.02!(B)$);
		
		\draw[-, thick] (A) -- (B);
		%\node at (A) {$($};
		%\node at (B) {$)$};
		%\filldraw [black,fill=white] (A) circle (3pt);
		\filldraw [black] (e) circle (1pt) node[above] {$\ens$};
		\filldraw [black] (a) circle (1pt) node[above] {$\ens[a]$};
		\filldraw [black] (b1) circle (1pt) node[above] {$\ens[b]_1$};
		\filldraw [black] (b2) circle (1pt) node[above] {$\ens[b]_2$};
		\filldraw [black] (b3) circle (1pt);
		\filldraw [black] (b4) circle (1pt) node[above] {$\cdots$};
		\filldraw [black] (b5) circle (1pt);
	\end{tikzpicture}
	%\includegraphics[width=0.4\textwidth]{tempimages/CapacitySupremum.jpg}
\end{figure}
	
	\begin{remark}
		We need to take the supremum as the maximum may not exist. For example, as shown in the figure, consider a discrete classical ensemble space and remove the extreme points. At the moment, there is no reason to rule out such space as unphysical.
	\end{remark}
	
	\begin{coro}
		Let $\ens, \ens[a] \in \Ens$, then $\fraction_{\ens}(\ens[a]) = 0$ if and only if $\ens[a]$ is not a component of $\ens$.
	\end{coro}
	
	\begin{proof}
		Note that $\ens[a]$ is a component of $\ens$ if we can write $\ens = \lambda \ens[a] + \bar{\lambda} \ens[b]$ for some $\lambda \in (0,1]$ and $\ens[b] \in \Ens$. In this case, $\fraction_{\ens}(\ens[a]) \geq \lambda > 0$. Conversely, if $\fraction_{\ens}(\ens[a]) > 0$ we can find $\lambda \in (0,\fraction_{\ens}(\ens[a])]$ such that $\ens = \lambda \ens[a] + \bar{\lambda} \ens[b]$ for some $\ens[b] \in \Ens$.
	\end{proof}
\end{mathSection}

\begin{figure}[H]
	\centering
	\begin{tikzpicture}		
		%\draw [help lines] (0,0) grid (10,3);
		
		\coordinate (center) at (1.5,1.5);
		\def\radius{1.25}
		\coordinate (e) at ($(center)+\radius*({cos(75)},{sin(75)})$);
		\coordinate (a1) at ($(center)+\radius*({cos(-65)},{sin(-65)})$);
		\coordinate (a2) at ($(center)+\radius*({cos(10)},{sin(10)})$);
		\coordinate (a3) at ($(center)+\radius*({cos(48)},{sin(48)})$);
		\coordinate (a4) at ($(center)+\radius*({cos(62)},{sin(62)})$);
		\coordinate (a5) at ($(center)+\radius*({cos(70)},{sin(70)})$);
		\draw [fill=ensembleFill, thick] (center) circle (\radius);
		\filldraw [black] (e) circle (1pt) node[above] {$\ens$};
		\filldraw [black] (a1) circle (1pt) node[below] {$\ens[a]_1$};
		\filldraw [black] (a2) circle (1pt) node[right] {$\ens[a]_2$};
		\filldraw [black] (a3) circle (1pt) node[right] {$\ens[a]_3$};
		\filldraw [black] (a4) circle (1pt);
		\filldraw [black] (a5) circle (1pt);
	\end{tikzpicture}
	\caption{Discontinuity of fraction.}\label{pm-es-fig-fractionDiscontinuous}
\end{figure}

Intuitively, we would expect the fraction $\fraction_{\ens}(\ens[a])$ to be continuous both in $\ens[a]$ and $\ens$, but this is not the case. For example, as shown in Fig. \ref{pm-es-fig-fractionDiscontinuous}, let $\Ens$ be the Bloch ball and let $\ens[a]_i$ be a sequence of pure states (i.e. extreme points) that converge to another pure state $\ens$. Since they are all extreme points, we have $\fraction_{\ens}(\ens[a]_i) = 0$ and $\fraction_{\ens[a]_i}(\ens) = 0$ for all $i$. However, $\fraction_{\ens}(\ens) = 1$. This is the recurring problem of the non-continuity at the edges. The following conjecture, however, may still be true:
\begin{conj}
	The fraction $\fraction_{\ens}(\ens[a])$ is a continuous function both in $\ens[a]$ and $\ens$ in the algebraic interior.
\end{conj}


\begin{figure}[h]
	\centering
	\def\xstep{0.75em}
	\def\bHeight{0.6}
	\def\tHeight{0.1}
	\def\tGap{0.3}
	\begin{tikzpicture}
		%\draw [help lines] (0,0) grid (10,3);
		
		\node at (1,0.5) {$\ens_{123456}$};
		
		\draw (0, 0) foreach \x in {1,...,6} { -- (\tGap * \x, 0) -- (\tGap * \x, -\tHeight) (\tGap * \x, 0) } -- ++(\tGap, 0);
		\draw plot[ybar, bar width={\tGap cm / 2}] coordinates{(\tGap * 1, \bHeight/6) (\tGap * 2, \bHeight/6) (\tGap * 3, \bHeight/6) (\tGap * 4, \bHeight/6) (\tGap * 5, \bHeight/6) (\tGap * 6, \bHeight/6)};
		
		\node at (2.6,0) {$ = \frac{1}{3}$};
		
		\node at (4.2,0.6) {$\frac{1}{2} \ens_3 + \frac{1}{2} \ens_4$};
		
		\draw (3, 0) foreach \x in {1,...,6} { -- (3 + \tGap * \x, 0) -- (3 + \tGap * \x, -\tHeight) (3 + \tGap * \x, 0) } -- ++(\tGap, 0);
		\draw plot[ybar, bar width={\tGap cm / 2}] coordinates{(3 + \tGap * 3, \bHeight/2)(3 + \tGap * 4, \bHeight/2)};
		
		\node at (5.6,0) {$ + \frac{2}{3}$};
		
		\node at (7,0.5) {$\ens_{1256}$};
		
		\draw (6, 0) foreach \x in {1,...,6} { -- (6 + \tGap * \x, 0) -- (6 + \tGap * \x, -\tHeight) (6 + \tGap * \x, 0) } -- ++(\tGap, 0);
		\draw plot[ybar, bar width={\tGap cm / 2}] coordinates{(6 + \tGap * 1, \bHeight/4) (6 + \tGap * 2, \bHeight/4)  (6 + \tGap * 5, \bHeight/4) (6 + \tGap * 6, \bHeight/4)};
	\end{tikzpicture}
	\caption{Visual representation of the fraction capacity.}\label{pm-es-fig-fractionCapacity}
	%\includegraphics[width=0.6\textwidth]{tempimages/FractionCapacity.jpg}
\end{figure}

We now define \textbf{fraction capacity} of a set of ensembles with respect to another ensemble. Like before, as shown in Fig. \ref{pm-es-fig-fractionCapacity}, suppose $\ens_{123456}$ is a uniform distribution for a six-faced die, but now take $A = \{ \ens_3, \ens_4 \}$ as, respectively, the outcomes $3$ and $4$ with $100\%$ probability. Then $\ens$ can be understood as $\ens_{123456} = \frac{1}{3} \left(\frac{1}{2} \ens_3 + \frac{1}{2} \ens_4 \right) + \frac{2}{3} \ens_{1256}$, where $\ens_{1256}$ is the uniform distribution over outcomes 1,2,5 and 6. Again, note that $\frac{1}{3}$ is the highest coefficient we can put in front of any convex combination of elements of $A$, and still make a convex combination that has $\ens$ as a result. That is how we define the fraction capacity of $A$ with respect to $\ens_{123456}$.

The fraction capacity, then, defines how much of the ensemble $\ens$ can be constructed with elements of $A$. The term capacity is used first because, intuitively, it tells us how much $A$ can hold, and second because capacity is a name used to describe non-additive measures. The fraction capacity, in fact, has several nice properties. First, its value is always between zero and one, since the coefficient of a convex combination must be so bound. Second, it is monotone in the sense that if $A$ gets bigger, the fraction capacity cannot decrease. Third, it is subadditive, meaning that the fraction capacity of the union of two sets must be the sum of the respective fraction capacities or less. Fourth, it is continuous from below. Note that if subadditivity is replaced by additivity, these are exactly the defining properties of a probability measure, since additivity and continuity are equivalent to $\sigma$-additivity.

\begin{mathSection}
	\begin{defn}
		Let $\ens \in \Ens$ be an ensemble and $A \subseteq \Ens$ a subset. The \textbf{fraction capacity} of $A$ with respect to $\ens$ is the biggest fraction achievable with convex combinations of $A$. That is, $\fcap_{\ens}(A) = \sup(\fraction_{\ens}(\hull(A))\cup\{0\})$.
	\end{defn}

	\begin{coro}
		The fraction capacity is a convex supremum of the fraction with $0$ for the empty set.
	\end{coro}
	
	\begin{remark}
		Since we have shown that the fraction is not a continuous function, we can use the same example to show that the fraction capacity of the closed hull is not necessarily the same as the fraction capacity of the convex hull. As previously shown in Fig. \ref{pm-es-fig-fractionDiscontinuous}, let $\Ens$ be a Bloch ball and let $A=\{\ens[a]_i\}$ be a sequence of pure states (i.e. extreme points) that converge to another pure state $\ens \notin \{\ens[a]_i\}$. Since $\ens$ is an extreme point, it can only be written as a convex combination of itself. Since $\ens \notin A$, $\ens \notin \hull(A)$ and therefore $\fcap_{\ens}(\hull(A))=0$. However, since $\ens$ is the limit of the sequence, we have $\ens \in \chull(A)$, and therefore $\fcap_{\ens}(\chull(A)) = 1$. The fraction capacity of the closed hull, then, can be different from the fraction capacity of the convex hull.
		
		An open question is whether some type of equality still holds for internal points and the $\sigma$-hull of internal points. For example, $\fcap_{\ens}(\hull(A)) = \fcap_{\ens}(\shull(A))$ if $\ens$ is an internal point and $A$ is a set of internal points.
	\end{remark}

	\begin{coro}
		The fraction capacity uniquely identifies an ensemble. That is, let $\ens[a], \ens[b] \in \Ens$ such that $\ens[a] \neq \ens[b]$. Then $\fcap_{\ens[a]} \neq \fcap_{\ens[b]}$.
	\end{coro}
	
	\begin{proof}
		Note that $\ens = 1 \ens[a] + 0 \ens[b]$ if and only if $\ens = \ens[a]$. Therefore, let $\ens[a], \ens[b] \in \Ens$ such that $\ens[a] \neq \ens[b]$. We have $\fcap_{\ens[a]}(\{\ens[a]\}) = 1$ and $\fcap_{\ens[b]}(\{\ens[a]\}) \neq 1$. Which means $\fcap_{\ens[a]} \neq \fcap_{\ens[b]}$.
	\end{proof}
	
	\begin{prop}
		The fraction capacity with respect to an ensemble is a set function that is
		\begin{enumerate}
			\item non-negative and unit bounded: $\fcap_{\ens}(A) \in [0,1]$
			\item increasing: $A \subseteq B \implies \fcap_{\ens}(A) \leq \fcap_{\ens}(B)$
			\item subadditive: $\fcap_{\ens}(A \cup B) \leq \fcap_{\ens}(A) + \fcap_{\ens}(B)$
			\item continuous from below: $\fcap_{\ens}(\lim\limits_{i \to \infty} A_i) = \lim\limits_{i \to \infty} \fcap_{\ens}(A_i)$ for any increasing sequence $\{A_i\}$
		\end{enumerate}
	\end{prop}
	
	\begin{proof}
		1. By \ref{pm-es-convexSupremumProps}, $\fcap_{\ens}(A) \in [0, \sup(\fraction_{\ens}(\Ens))] = [0,1]$. 
	
		2. By \ref{pm-es-convexSupremumProps}.
		
		3. Let $A, B \subseteq \Ens$ and let $p \in [0,1]$ such that $\ens = p \ens_1 + \bar{p} \ens_2$ for some $\ens_1 \in \hull(A \cup B)$ and $\ens_2 \in \Ens$. Since $\ens_1 \in \hull(A \cup B)$, we can write $\ens_1 = \lambda \ens[a] + \bar{\lambda} \ens[b]$ for some $\lambda \in [0,1]$, $\ens[a] \in A$ and $\ens[b] \in B$. Therefore we have $\ens = p \lambda \ens[a] + p \bar{\lambda} \ens[b] + \bar{p} \ens_2$. By the definition of fraction capacity, we must have $p\lambda \leq \fcap_{\ens}(A)$ and $p\bar{\lambda} \leq \fcap_{\ens}(B)$, therefore $p = p\lambda + p\bar{\lambda} \leq \fcap_{\ens}(A) + \fcap_{\ens}(B)$. 	Since $\fcap_{\ens}(A \cup B)$ is the supremum for a set of coefficients $p$ for which the expression always holds, we have $\fcap_{\ens}(A \cup B) \leq \fcap_{\ens}(A) + \fcap_{\ens}(B)$. The fraction capacity is subadditive.
	
		4. By \ref{pm-es-convexSupremumProps}.
	\end{proof}
	

\begin{remark}
	The fraction capacity is, in general, not continuous from above. Let $\Ens$ be the Bloch ball. Let $\ens \in \Ens$ be the maximally mixed state (i.e. the center of the sphere). Let $\{\ens_i\}$ be a countable set of distinct pure states. Let $A_j = \{ \ens_i \, | \, i \geq j\}$. Then $\fcap_{\ens}(A_j) \geq \frac{1}{2}$ for all $j$ because $A_j$ contains at least one pure state. This means that $\lim\limits_{i \to \infty} \fcap_{\ens}(A_i) \geq \frac{1}{2}$. However, $\bigcap A_j = \emptyset$ which means $\fcap_{\ens}(\lim\limits_{i \to \infty} A_i) = 0$. This means that, in general, $\fcap_{\ens}(\lim\limits_{i \to \infty} A_i) \neq \lim\limits_{i \to \infty} \fcap_{\ens}(A_i)$ for a decreasing sequence.
\end{remark}
	
\end{mathSection}

One open question is whether subadditivity extends to the infinite case. Since additivity plus continuity implies $\sigma$-additivity\footnote{See for example \href{https://link.springer.com/book/10.1007/978-3-319-30690-2}{Michel Grabisch, Set Functions, Games and Capacities in Decision Making}.}, it may be true that subadditivity plus continuity implies $\sigma$-subadditivity.

\begin{conj}
	The fraction capacity is $\sigma$-subadditive. That is, $\fcap_{\ens}(\bigcup A_i) \leq \sum_i \fcap_{\ens}(A_i)$.
\end{conj}


It is still an open question to understand exactly when the fraction capacity recovers additivity. We believe that this should be related to the ability to write an element as a mixture of separate components.

\begin{conj}
	The fraction capacity $\fcap_{\ens}$ is additive over $A, B \subseteq \Ens$ if and only if there exists a $C \subseteq \Ens$ such that $A \separate B$, $B \separate C$, $A \separate C$ and $\ens \in \hull(A\cup B\cup C)$.
\end{conj}

We now show that fraction capacity recovers classical probability in the classical case. The key insight is that each event, each Borel set $A$, corresponds to a subspace of probability measures for which that event will be true. This is the set of probability measure whose support is always within the given Borel set $A$. A probability measure, then, can always be decomposed in a part for which that event will be true, the part supported by $A$, and a part for which that event will be false, the part supported by the complement. The fraction capacity corresponds to the mixing coefficient of this decomposition.

\begin{figure}[H]
	\centering
	\def\xstep{0.75em}
	\def\bHeight{0.6}
	\def\tHeight{0.1}
	\def\tGap{0.3}
	\begin{tikzpicture}
		%\draw [help lines] (0,0) grid (10,3);
		
		\def\pa{1/13}
		\def\pb{2/13}
		\def\pc{3/13}
		\def\pd{3/13}
		\def\pe{2/13}
		\def\pf{2/13}
		
		\node at (1,0.5) {$p$};
		
		\draw (0, 0) foreach \x in {1,...,6} { -- (\tGap * \x, 0) -- (\tGap * \x, -\tHeight) (\tGap * \x, 0) } -- ++(\tGap, 0);
		\draw plot[ybar, bar width={\tGap cm / 2}] coordinates{(\tGap * 1, \bHeight * \pa) (\tGap * 2, \bHeight * \pb) (\tGap * 3, \bHeight * \pc) (\tGap * 4, \bHeight * \pd) (\tGap * 5, \bHeight * \pe) (\tGap * 6, \bHeight * \pf)};
		
		\node at (2.8,0) {$ = p(A)$};
		
		\def\pos{3.4}
		\def\pa{1/9}
		\def\pb{2/9}
		\def\pc{3/9}
		\def\pd{3/9}
		\def\pe{0}
		\def\pf{0}
		\node at (4.6,0.5) {$p_A$};
		
		\draw (\pos, 0) foreach \x in {1,...,6} { -- (\pos + \tGap * \x, 0) -- (\pos + \tGap * \x, -\tHeight) (\pos + \tGap * \x, 0) } -- ++(\tGap, 0);
		\draw plot[ybar, bar width={\tGap cm / 2}] coordinates{(\pos + \tGap * 1, \bHeight * \pa) (\pos + \tGap * 2, \bHeight * \pb) (\pos + \tGap * 3, \bHeight * \pc) (\pos + \tGap * 4, \bHeight * \pd) (\pos + \tGap * 5, \bHeight * \pe) (\pos + \tGap * 6, \bHeight * \pf)};
		%\draw plot[ybar, bar width={\tGap cm / 2}] coordinates{(3 + \tGap * 3, \bHeight/2)(3 + \tGap * 4, \bHeight/2)};
		
		\node at (6.25,0) {$ + p(A^{\complement})$};
		
		
		\def\pos{6.95}
		\node at (8.15,0.5) {$p_{A^{\complement}}$};
		\def\pa{0}
		\def\pb{0}
		\def\pc{0}
		\def\pd{0}
		\def\pe{2/4}
		\def\pf{2/4}
		
		\draw (\pos, 0) foreach \x in {1,...,6} { -- (\pos + \tGap * \x, 0) -- (\pos + \tGap * \x, -\tHeight) (\pos + \tGap * \x, 0) } -- ++(\tGap, 0);
		\draw plot[ybar, bar width={\tGap cm / 2}] coordinates{(\pos + \tGap * 1, \bHeight * \pa) (\pos + \tGap * 2, \bHeight * \pb) (\pos + \tGap * 3, \bHeight * \pc) (\pos + \tGap * 4, \bHeight * \pd) (\pos + \tGap * 5, \bHeight * \pe) (\pos + \tGap * 6, \bHeight * \pf)};
		
		%\draw (6, 0) foreach \x in {1,...,6} { -- (6 + \tGap * \x, 0) -- (6 + \tGap * \x, -\tHeight) (6 + \tGap * \x, 0) } -- ++(\tGap, 0);
		%\draw plot[ybar, bar width={\tGap cm / 2}] coordinates{(6 + \tGap * 1, \bHeight/4) (6 + \tGap * 2, \bHeight/4)  (6 + \tGap * 5, \bHeight/4) (6 + \tGap * 6, \bHeight/4)};
	\end{tikzpicture}
	\caption{Connection between probability and fraction capacity.}
\end{figure}


\begin{mathSection}
	\begin{prop}
		Let $\Ens$ be a discrete or continuous classical ensemble space with sample space $X$. Let $A \in \Sigma_X$ be an event and $\Ens_A \subseteq \Ens$ the set of distributions with support contained in $A$. Let $p \in \Ens$ a probability measure in the ensemble space. Then $p(A) = \fcap_{p}(\Ens_A)$. That is, the probability of an event is equal to the fraction capacity of the subspace corresponding to that event.
	\end{prop}
	\begin{proof}
		Let $p \in \Ens$ be a probability measure over $X$, absolutely continuous with respect to the appropriate measure $\mu$ (i.e. discrete for the discrete case and Liouville for the continuous case). Given $A \in \Sigma_X$, we can write $p = p(A) p_A + p(A^{\complement}) p_{A^{\complement}}$ where  $p_A$ is a probability measure with support with $A$ and $p_{A^{\complement}}$ is a probability measure with support with $A^{\complement}$.
		
		Let $\Ens_A \subseteq \Ens$ be the subset of probability measures with support contained in $A$. Then $p_A \in \Ens_A$ and $\fcap_{p}(\Ens_A) \geq p(A)$. Given that $p_{A^{\complement}}$ has support disjoint from $A$, we can find no further component of $p$ that has support in $A$. Therefore $\fcap_{p}(\Ens_A) = p(A)$. The fraction capacity, then, recovers the probability.
	\end{proof}
\end{mathSection}
	
We now show that the fraction capacity recovers probability over measurements in the quantum case. The setup is essentially the same that as of the classical case, except that we start with an observable $O$, and each Borel set $A$ for that observable will have a corresponding subspace of $\mathcal{H}$ and a projector $\Pi_A$. A post-measurement state of $O$ is a mixed state that commutes with $O$, since it will commute with any projector that commutes with $O$. We can show, then, that the fraction capacity recovers the decomposition of any post-measurement state of $O$ over the subspace corresponding to $\Pi_A$.

\begin{mathSection}
	\begin{prop}
		Let $\Ens$ be a quantum ensemble space associated to a Hilbert space $\mathcal{H}$. Let $O : \mathcal{H} \to \mathcal{H}$ be a quantum observable. Let $A \in \Sigma_{\mathbb{R}}$ be a Borel set and $\Pi_{A}$ be the projector associated with the subspace of $\mathcal{H}$ corresponding to the subset $A$ of the spectrum of $O$. Let $\Ens_A$ be the set of ensembles within that subspace. Let $\rho \in \Ens$ be a mixed state that commutes with $O$ (i.e. the ensemble resulting from a measurement of $O$). Then $\tr(\rho \Pi_A) = \fcap_{\rho}(\Ens_A)$.
	\end{prop}
	
	\begin{proof}
		Let $\rho \in \Ens$ be a mixed state that commutes with $O$. Let $A \in \Sigma_{\mathbb{R}}$ be a Borel set and $\Pi_{A}$ be the projector associated with the subspace of $\mathcal{H}$ corresponding to the subset $A$ of the spectrum of $O$. Then we can write $\rho = \lambda \rho_A + \bar{\lambda} \rho_{A^{\complement}}$ where $\rho_A, \rho_{A^{\complement}} \in \Ens$ are such that $\tr(\rho_A \Pi_A) = 1$ and  $\tr(\rho_{A^{\complement}} \Pi_A) = 0$. We have $\tr(\rho \Pi_A) = \lambda$.
		
		Let $\Ens_A \subseteq \Ens$ be the subset of mixed states $d$ such that $\tr(d \Pi_A) = 1$. Then $d \in \Ens_A$ and $\fcap_{\rho}(\Ens_A) \geq \lambda$. Given that $\rho_{A^{\complement}}$ is in the subspace orthogonal to $\Ens_A$, we can find no further component of $\rho$ in that subspace. Therefore $\fcap_{\rho}(\Ens_A) = \lambda$. We have $\tr(\rho \Pi_A) = \lambda = \fcap_{\rho}(\Ens_A)$ The fraction capacity, then, recovers the probability.
	\end{proof}
\end{mathSection}

\section{Statistical properties and quantities}

In this section we define statistical quantities, which are the generalization of classical random variables and quantum observables. Given our ensemble-first approach, statistical quantities are affine (linear) functions of ensembles as they represent an expectation over ensembles. The possible values of the quantities will need to be recovered with constructions that mimic spectral theory, though we will not want to talk about eigenstates and eigenvalues in the general case since these are not properly defined for a continuous quantity.

\subsection{Properties and quantities}

These definitions extend the notion of properties and quantities that we defined for an experimental domain. In that case, we simply required that a property be a continuous map to a set of possible values for the property. For a statistical property, the set of values are a topological convex space, so that the property allows statistical mixing. For a quantity, the topological convex space is simply a linearly ordered property, which in this case will always be a real-valued quantity. The convexity, the averaging operation, fills in the gaps in the linear order in case that all intermediate values can be obtained. For example, even if the values on the pure states are rationals or integers, the expectations will always span a contiguous interval of the reals.

\begin{mathSection}
	
\begin{defn}
	A \textbf{statistical property}, or simply property, is an attribute that allows statistical mixing. Formally, it is a continuous map $F : \Ens \to \mathcal{Q}$ where $\mathcal{Q}$ is a convex topological space such that $F(p \ens[a] + \bar{p} \ens[b]) = p F(\ens[a]) + \bar{p} F(\ens[b])$ (i.e. it is an affine map).
	
	A \textbf{statistical quantity}, or statistical variable, or simply variable, is a numerical statistical property. That is, it is a continuous real-valued affine map $F : \Ens \to \mathbb{R}$.
\end{defn}

\begin{justification}
	This definition extends to the statistical case the general definition of properties and quantities we already gave in the previous chapter. As before, continuity is required since verifying the value of the quantity corresponds to verifying that we are dealing with a specific subset of ensembles.
	
	The ability to create convex combinations corresponds to the ability to create statistical averages. Therefore $\mathcal{Q}$ must be a convex set. Given that preparation instances are assumed to be independent, a mixture of the preparations will produce a mixture of the properties according to the same fraction coefficients.
	
	Quantities are simply properties that are linearly ordered. The ability to create convex combinations becomes the ability to take weighted averages of the quantities. Regardless of whether one starts from integers, rationals or real-valued quantities, the statistical averages will, in general, be a real number.
	
	Note that the numerical value cannot be infinite. Given that we can only measure the finite average of a finite sequence of outcomes, we must have that these sampled averages converge to the expectation. The only way this could happen for an infinite expectation would be if the sampled average kept increasing over time. But this would be a contradiction of the assumption that an ensemble represents a reproducible collection of preparations. Therefore, statistical variables with infinite expectations are not justified.
\end{justification}

\begin{remark}
	Note that variables will have a contiguous range on the ensembles because, given two ensembles with different values, we can mix them to obtain any intermediate value. This does not mean that the variable can take all possible values on pure states. For example, the number of particles in a pure state will necessarily be a non-negative integer, but we can mix those to create ensembles that have non-integer average number of particles.
\end{remark}

\begin{coro}
	A set of statistical quantities $\{F_i\}_{i \in I}$ can be collected into a single statistical property $F : \Ens \to \mathbb{R}^I$ where $F(\ens) = \{F_i(\ens)\}_{i \in I}$.
\end{coro}
\end{mathSection}

Here we show that statistical quantities recover the standard random variables of classical mechanics. That is, every statistical variable is a random variable and every random variable is a statistical variable.

\begin{mathSection}
\begin{prop}
	Let $\Ens$ be a discrete or continuous classical ensemble space. Then each statistical quantity is the expectation of a random variable and vice-versa.
\end{prop}

\begin{proof}
	Let $\Ens$ be a discrete or continuous classical ensemble space over some sample space $X$. Let $f : X \to \mathbb{R}$ be a random variable and $p \in \Ens$ a probability measure, then the expectation $F(p) \mapsto \int_{X} f dp$ is a continuous real affine map and is therefore a statistical quantity. Conversely, let $F : \Ens \to \mathbb{R}$ be a linear functional of the measures. Then, by the Riesz representation theorem, we can write $F(p) = \int_{X} f dp$ for some $f : X \to \mathbb{R}$.
\end{proof}
\end{mathSection}

For a quantum system, we can show that all expectations of observables are statistical quantities.

\begin{mathSection}
\begin{prop}
	Let $\Ens$ be a quantum ensemble space. All expectations of observables are statistical quantities.
\end{prop}

\begin{proof}
	Let $\Ens$ be a quantum ensemble space over some Hilbert space $\mathcal{H}$. Let $O : \mathcal{H} \to \mathcal{H}$ be a Hermitian operator and $\rho$ be a density matrix. Then $F(\rho) \mapsto \tr(\rho O)$ is a continuous real affine map of $\Ens$ and it is therefore a statistical quantity.
\end{proof}

\begin{remark}
	We still need to prove that the converse is true: every statistical quantity in quantum mechanics corresponds to an operator. We may need to constrain the problem to bounded quantities, and recover the unbounded ones only as a limit.
\end{remark}
\end{mathSection}


\subsection{Quantifiable spaces and locally convex vector spaces}

In physics, we typically are able to fully identify ensembles by using measurable quantities. An ensemble space is quantifiable, then, when we have enough statistical quantities to recover all ensembles. There is a subtle problem with infinity, though, that is still not completely closed.

In principle, the space of ensembles may include quantities whose expectation is infinite over some distributions. In classical mechanics, for example, given any probability distribution $\rho$ with infinite support we can find a random variable $f$ such that $\int_X f \rho dx$ diverges. For example, we can simply set $f(x) = 1/\rho(x)$ where $\rho(x)\neq0$ and $f(x) = 0$ otherwise. This can be solved in classical mechanics because we can always find distributions with compact support, which will guarantee convergence for all functions. In quantum mechanics, however, a wavefunction with compact support in position will have non-compact support in momentum and vice-versa. However, if we were to restrict ourselves to the Schwartz wavefunctions we would have finite expectation for all polynomials of position and momentum, and they would be dense in the Hilbert space. What makes sense to expect physically, then, is that there is a set of quantities that physically define the system which will need finite expectation. These will need to remain finite under coordinate transformations and time evolution for the physics to make sense. All these details are yet to be understood and framed in a mathematically precise way. Yet, it is clear that we need, at least, to characterize the case where a set of statistical quantities fully characterizes all ensembles.

\begin{mathSection}
\begin{defn}
	A \textbf{quantifiable} ensemble space is an ensemble space where each ensemble can be identified by a set of statistical quantities. That is, there is family of statistical variables $F_i : \Ens \to \mathbb{R}$ such that, given $\ens_1, \ens_2 \in \Ens$, $F_i(\ens_1) = F_i(\ens_2)$ for all $i$ if and only if $\ens_1 = \ens_2$. Moreover, the topology is generated by those quantities.
\end{defn}
\end{mathSection}

We are going to show that the statistical quantities can be used to construct seminorms on the embedding vector space inducing a topology that is Hausdorff and locally convex. Since the statistical quantities are continuous in both the topology of the ensemble space and on this topology of the embedding vector space, the embedding of the ensemble space is continuous. Note that this embedding is not necessarily a homeomorphism onto the image as convergence of the expectation of continuous variables (i.e. weak convergence) does not guarantee convergence of the entropy (which requires, for example, convergence of the related inner products in the classical and quantum cases). Therefore, all the topological details of the ensemble space remain open even in this case.

\begin{mathSection}
\begin{prop}
	A statistical variable $F$ induces a seminorm on the vector space that embeds $\Ens$.
\end{prop}

\begin{proof}
	Let $F$ be a variable on $\Ens$, let $\ens[a] \in \Ens$ be an internal point and let $V_{\ens[a]}$ be the vector space of differences. Define $F_{\ens[a]} : V_{\ens[a]} \to \mathbb{R}$ such that $F_{\ens[a]}([r(\ens[b]-\ens[a])]) = |r (F(\ens[b]) - F(\ens[a]))|$.
	
	First we show that $F_{\ens[a]}$ does not depend on the representative. If we have the zero class, then either $r=0$ or $\ens[b] = \ens[a]$. In both cases, the function evaluates to zero. For the non-zero class, we have
	\begin{equation}
		\begin{aligned}
			F_{\ens[a]}\left(\left\{(r+j)\left(\left(\frac{r}{r+j}\ens[b] +\frac{j}{r+j} \ens[a]\right) -\ens[a]\right)\right\}\right) &= \left\{ \left|(r+j)\left(F\left(\frac{r}{r+j}\ens[b] +\frac{j}{r+j} \ens[a]\right) - F(\ens[a]) \right)\right| \right\} \\
			&= \left\{ \left|(r+j)\left(\frac{r}{r+j}F(\ens[b]) +\frac{j}{r+j} F(\ens[a]) - F(\ens[a]) \right)\right| \right\} \\
			&= \left\{ \left| r F(\ens[b]) + j F(\ens[a]) - (r+j) F(\ens[a]) \right| \right\} \\
			&= \left\{|r (F(\ens[b]) - F(\ens[a]))|\right\}.
		\end{aligned}
	\end{equation}
	
	Since $F_{\ens[a]}$ evaluates to an absolute value, it is a non-negative function. Since $F_{\ens[a]}(r[s(\ens[b]-\ens[a])]) = F_{\ens[a]}([(rs)(\ens[b]-\ens[a])]) = |(rs) (F(\ens[b]) - F(\ens[a]))| = |r| |s (F(\ens[b]) - F(\ens[a]))| = |r| F_{\ens[a]}([s(\ens[b]-\ens[a])])$, $F_{\ens[a]}$ is absolutely homogeneous. We also have
	\begin{equation}
	\begin{aligned}
		&F_{\ens[a]}([r(\ens[b]-\ens[a])] + [s(\ens[c]-\ens[a])])  \\
		&= F_{\ens[a]}\left(\left[(r+s+k) \left(\left(\frac{r}{r+s+k}\ens[b] + \frac{s}{r+s+k} \ens[c] +\frac{k}{r+s+k} \ens[a] \right) - \ens[a]\right)\right]\right) \\
		&= \left|(r+s+k)\left(F\left(\frac{r}{r+s+k}\ens[b] + \frac{s}{r+s+k} \ens[c] +\frac{k}{r+s+k} \ens[a]\right) - F(\ens[a]) \right)\right| \\
		&= \left|r F(\ens[b]) + s F(\ens[c]) + k F(\ens[a]) - (r+s+k) F(\ens[a]) \right| \\
		&= \left|r ( F(\ens[b]) - F(\ens[a]))  + s ( F(\ens[c]) - F(\ens[a])) \right| \\
		&\leq \left| r ( F(\ens[b]) - F(\ens[a])) \right| + \left| s ( F(\ens[c]) - F(\ens[a])) \right| \\
		&= F_{\ens[a]}([r(\ens[b]-\ens[a])]) + F_{\ens[a]}([s(\ens[c]-\ens[a])]).
	\end{aligned}
	\end{equation}
	This means that $F_{\ens[a]}$ is subadditive. Therefore $F_{\ens[a]}$ is a seminorm by \href{https://en.wikipedia.org/wiki/Seminorm}{definition}.
\end{proof}

\begin{prop}
	A quantifiable ensemble space embeds continuously into a Hausdorff locally convex topological vector space.
\end{prop}

\begin{proof}
	Let $\Ens$ be a quantifiable ensemble space and $V_{\ens[a]}$ be the embedding vector space corresponding to the internal point $\ens[a] \in \Ens$. Consider all the seminorms defined by the family of statistical variables that make $\Ens$ quantifiable. These will induce a topology on $V_{\ens[a]}$ making it a topological vector space (see \href{https://personal.math.ubc.ca/~cass/research/pdf/TVS.pdf}{Prop 2.2 in here}). Since the topology is generated by seminorms, it is locally convex. Since the seminorms fully identify each point, the topology is Hausdorff (see \href{https://personal.math.ubc.ca/~cass/research/pdf/TVS.pdf}{Prop 2.6 in here}).
	
	Now consider $\Ens$ as embedded into $V_{\ens[a]}$. If $F : \Ens \to \mathbb{R}$ is a statistical variable, then $|r F(\ens[b]) - F(\ens[a]) |$ is also a statistical variable since difference, scalar multiplication and absolute value are all continuous. This means that the open balls given by the seminorm on the vector space restricted to the ensemble space are open sets as they correspond to the open balls in the ensemble space. Moreover, since all elements of $F_i$, the statistical quantities that characterize the ensembles, are continuous in the ensemble space, a convergent sequence in the ensemble space will correspond to the convergence of all the seminorms in the vector space, thus convergence in the topology generated by said seminorms. The embedding, then, is continuous.
\end{proof}
\end{mathSection}

At this point, we are not guaranteed that ensembles can be characterized by a countable family of statistical variables. The issue is that, while a second countable locally convex topology is generated by countably many seminorms, we know that the topology of the ensemble space is second countable but we do not know whether the subtopology generated by the statistical variables through the seminorms is second countable. Physically, it would mean that we have a way to experimentally distinguish between two ensembles (i.e. a second countable topology) but not through the expectation of statistical quantities. The entropy would, for example, provide other more stringent ways. It is not yet clear whether this is something that can indeed happen, and therefore a potential source of new physics, or we simply are missing a way to prove the result. Currently, the second option seems more likely.

If one is able to prove that the topology induced by the statistical variables is second countable, or if we simply impose that a countable family of statistical variables is enough to distinguish between ensembles, then the space is a metrizable second countable locally convex topological vector space.

\begin{mathSection}
\begin{prop}
	If a quantifiable ensemble space is fully determined by countably many statistical variables, then it embeds continuously into a metrizable second countable locally convex topological vector space.
\end{prop}

\begin{proof}
	If a quantifiable ensemble space is characterized by countably many statistical variables, the topology induced on the vector space is second countable. A Hausdorff second countable topological vector space is metrizable.
\end{proof}

\begin{remark}
	Note that we are missing completeness in terms of the seminorms to obtain a Fr\'echet space and it is unclear we are going to have it. For the classical case, suppose we restrict ourselves to all absolutely continuous probability measures with compact support. Expectations of all polynomials of position and momentum fully identify the measure. However, Dirac measures are also fully identified by those quantities, which can be achieved as the limit of a uniform distribution whose support shrinks to a point. That would give us a sequence of convergent expectations whose corresponding ensembles do not converge in the space.
\end{remark}

\begin{prop}
	Discrete/continuous classical ensemble spaces and quantum ensemble spaces are quantifiable.
\end{prop}

\begin{proof}
	For discrete classical spaces, the expectation of the indicator of each extreme point defines a countable set of quantities that fully identifies the distribution.
	
	For continuous classical spaces, note that $L^1(\mathbb{R}^{2n})$ is a Hausdorff second countable locally convex topological space.
	
	For quantum ensemble spaces, the expectations of projectors define a family of statistical variables that can distinguish any pair of mixed states.
\end{proof}
\end{mathSection}

An open problem is whether statistical quantities are required by ensemble spaces or they are an additional requirement. If we were able to show that ensemble spaces always embed continuously into a locally convex second countable topological vector space, one may use the seminorms to define quantities. Another approach is to investigate whether the existence of linear transformations parameterized by real quantities, such as time evolution, gives us statistical quantities through the generators of the transformations. It would also mean that, if time is not modeled as a real-valued parameter, then the ensemble space would have to change.

\subsection{Connection to Choquet theory}

We now want to establish a connection between ensemble spaces and Choquet theory. Specifically, we want to show that the fraction capacity restricted to the extreme points corresponds to the supremum of all possible measures that represent an ensemble. Moreover, the fraction capacity over the extreme points is additive exactly when the ensemble space is a Choquet simplex.

The goal of Choquet theory is to study how a point of a compact convex set can be represented by a probability measure over its extreme points. First, we need to define what it means to represent an ensemble with a probability measure. Let $E$ be a convex subset of a locally convex topological vector space $V$ and $X$ the set of extreme points. We say that $\ens \in E$ is represented by a probability measure $p : \Sigma_{E} \to [0,1]$ if $F(\ens) = \int_E F dp$ for every continuous linear functional $F$ on $V$. Moreover, we say that $p$ is supported by the extreme points if $p(X) = 1$. Given any probability measure $p$ over the extreme points $X$, we can always find an $\ens \in E$ that is represented by $p$. Therefore, we have a map from the set of probability measures to the set of ensembles. Choquet theory characterizes how the map works in the opposite direction.

The first question is whether each ensemble is represented by a probability measure. Choquet theory tells us that this is the case:
\begin{mathSection}
\begin{thrm}[Choquet-Bishop-de Leeuw]\label{pm-es-ChoquetExistence}
	Let $E$ be a compact convex subset of a locally convex topological vector space $V$, and let $\ens \in E$. Then there exists a probability measure $p_{\ens}$ on $E$ which represents $\ens$ and is supported by the extreme points $X$.
\end{thrm}
\end{mathSection}
The second question is whether the representation is unique. Choquet theory has a clear answer.
\begin{mathSection}
\begin{thrm}[Choquet uniqueness]\label{pm-es-ChoquetUniqueness}
	Let $E$ be a compact convex subset of a locally convex topological vector space $V$. Then $E$ is a Choquet simplex if and only if for each $\ens \in E$ there exists a unique measure $p_{\ens}$ which represents $\ens$ and is supported by the extreme points $X$.
\end{thrm}
\end{mathSection}
The definition of a Choquet simplex is rather involved, so it is omitted. Intuitively, it is the generalization of the finite-dimensional simplex. The key point is that $E$ is a Choquet simplex if and only if the ensemble space does not allow multiple decompositions in terms of extreme points.

Note that there is a mismatch between ensemble spaces and the definition required by Choquet theory. While ensemble spaces are convex subsets of a vector space, they will not necessarily be compact. For example, in the classical continuous case, the space of absolutely continuous probability distributions is not compact, even if we restrict ourselves to a compact subset of phase space. The issue is that the extreme points, the Dirac distributions, are not ensembles. Moreover, ensemble spaces are equipped with an entropy, and therefore have more structure than required by Choquet theory.

However, the link we want to establish to Choquet theory is limited to the fraction capacity, which can be defined for any convex set. Therefore we are going to study the relationship between the fraction capacity in all cases where Choquet theory is applicable. If an ensemble space $\Ens$ is compact, the results will apply directly. If an ensemble space $\Ens$ admits some type of compactification $E$, like extending the space of ensembles to a compact set, then the results will apply to these extreme points that are not necessarily ensembles.

We first note that there is a connection between mixtures and the representation through a probability measure. We can, in fact, write $\ens = \lambda \ens[a] + \bar{\lambda} \ens[b]$ if and only if $\ens$ can be represented by the probability measure $p(\{\ens[a]\}) = \lambda$, $p(\{\ens[b]\}) = \bar{\lambda}$, and $p(\Ens \setminus \{\ens[a], \ens[b]\}) = 0$. Therefore, if $M_{\ens}$ is the set of probability measures that represent $\ens$, the fraction $\fraction_{\ens}(\ens[a])$ is given by the maximum value over $\{\ens[a]\}$ that can be taken by one of those measures. That is,  $\fraction_{\ens}(\ens[a]) = \sup \{ p(\{\ens[a]\}) \, | \, p \in M_{\ens} \}$.

\begin{mathSection}
\begin{prop}\label{pm-es-fractionSupChoquet}
	Let $E$ be a compact convex subset of a locally convex topological vector space $V$ and define the fraction capacity as for an ensemble space. Let $\ens \in E$ be a point and $M_{\ens}$ be the set of measures that represent $\ens$. Then $\fraction_{\ens}(\ens[a]) = \sup \{ p(\{\ens[a]\}) \, | \, p \in M_{\ens} \}$  for all $\ens[a] \in E$ and $\fcap_{\ens}(A) = \sup \{ p(A) \, | \, p \in M_{\ens} \}$ for all $A \subseteq E$.
\end{prop}

\begin{proof}
	Let $\ens \in E$ and suppose $\ens = \lambda \ens[a] + \bar{\lambda} \ens[b]$ with $\ens[a], \ens[b] \in E$ and $\lambda \in [0,1]$. Then the measure $p$ such that $p(\{\ens[a]\}) = \lambda$, $p(\{\ens[b]\}) = \bar{\lambda}$ and $p(E\setminus \{ \ens[a], \ens[b]\})=0$ represents $\ens$. In fact, for any statistical variable $F$ we have $F(\ens) = \lambda F(\ens[a]) + \bar{\lambda} F(\ens[b]) = \int_E F dp$. The converse is true as well. If $p \in M_{\ens}$ such that $p(\{\ens[a]\}) = \lambda$, $p(\{\ens[b]\}) = \bar{\lambda}$ and $p(E\setminus \{ \ens[a], \ens[b]\})=0$, then $\ens = \lambda \ens[a] + \bar{\lambda} \ens[b]$. Since a Dirac measure at a point $\ens[a]$ represents $\ens[a]$, the convex combination of two Dirac measures represents the convex combination of the two corresponding points. Now let $p \in M_{\ens}$ such that $p(\{\ens[a]\}) \neq 1$. We can write it as $p = p(\{\ens[a]\}) \delta_{\ens[a]} + p(E \setminus \{\ens[a]\})\frac{p|_{E \setminus \{\ens[a]\}}}{p(E \setminus \{\ens[a]\})}$ where $\delta_{\ens[a]}$ is the Dirac measure over $\ens[a]$. But since every probability measure will represent an element of $E$, we can find $\ens[b] \in E$ that is represented by $\frac{p|_{E \setminus \{\ens[a]\}}}{p(E \setminus \{\ens[a]\})}$. Therefore there is $\hat{p} \in M_{\ens}$ such that $\hat{p}(\{\ens[a]\}) = p(\{\ens[a]\})$, $\hat{p}(\{\ens[b]\}) = p(E \setminus \{\ens[a]\})$ and $\hat{p}(E\setminus \{ \ens[a], \ens[b]\})=0$. Which means $\fraction_{\ens}(\ens[a]) = \sup(\{ \lambda \in [0,1] \, | \, \exists \, \ens[b] \in E \text{ s.t. }  \ens = \lambda \ens[a] + \bar{\lambda} \ens[b] \}) = \sup \{ p(\{\ens[a]\}) \, | \, p \in M_{\ens} \}$.
	
	Now let $A \subseteq E$ be convex and let $M_A$ be the set of all measures whose support is in $A$. Each measure in $M_A$ will represent a point in $A$. Also, since the expectation of a convex combination of measures corresponds to a convex combination of the expectation values, $M_A$ will be convex as well. If $p$ represents $\ens$ and $p(A) \neq 0$, then $\frac{p|_A}{p(A)}$ is a probability measure that represents an element $\ens[a] \in A$ such that $\ens = p(A) \ens[a] + p(A^{\complement}) \ens[b]$. Therefore $\fraction_{\ens}(A) = \sup \{ p(A) \, | \, p \in M_{\ens} \}$.
\end{proof}
\end{mathSection}

If we restrict the fraction capacity to the set $X$ of extreme points, it will represent the supremum of all probability measures over the extreme points. If $E$ is a Choquet simplex, then each ensemble will be represented by a unique measure over the extreme points, therefore the fraction capacity over the extreme points will coincide with that measure, and will be additive. Conversely, if the fraction capacity over the extreme points is additive, the fraction capacity must be the supremum over a single measure. If the fraction capacity over the extreme points is additive for all ensembles, then each ensemble is represented by a unique probability measure over the extreme points. In other words, the additivity of the fraction capacity over the extreme points exactly captures the single decomposition of ensembles into pure states and the Choquet simplex case.

\begin{mathSection}
\begin{prop}
	Let $E$ be a compact convex subset of a locally convex topological vector space $V$. Then the following statements are equivalent:
	\begin{enumerate}
		\item $E$ is a Choquet simplex
		\item each ensemble is uniquely represented by a measure over its extreme points $X$
		\item the fraction capacity restricted over the extreme points is additive
	\end{enumerate}
\end{prop}

\begin{proof}
	By theorem \ref{pm-es-ChoquetUniqueness}, 1 and 2 are equivalent. 
	
	Now consider $\fraction_{\ens}|_X$. By \ref{pm-es-fractionSupChoquet}, $\fcap_{\ens}|_X$ gives us the supremum of all possible measures supported by $X$ that represent $\ens$. By \ref{pm-es-ChoquetExistence}, $\ens$ is represented by at least one measure. Suppose that it is represented by exactly one measure $p$. Then $\fraction_{\ens}|_X = p$ and therefore $\fraction_{\ens}|_X$ is additive. Now, suppose that $\ens$ is represented by at least two distinct measures $p_1$ and $p_2$. Then there will be a set $A \subseteq X$ such that $p_1(A) \neq p_2(A)$. Suppose, without loss of generality, that $p_1(A) > p_2(A)$. We will also have $p_1(A^{\complement}) = 1- p_1(A) < 1 - p_2(A) = p_2(A^{\complement})$. Therefore $\fcap_{\ens}(A \cup A^{\complement})=\fcap_{\ens}(X) = 1 = p_1(A) + p_1(A^{\complement}) < p_1(A) + p_2(A^{\complement}) \leq \fcap_{\ens}(A) + \fcap_{\ens}(A^{\complement})$. The fraction capacity is therefore additive if and only if $\ens$ is represented by a single measure over the extreme points. Therefore 2 and 3 are equivalent.
\end{proof}
\end{mathSection}

\subsection{Macrostates and thermodynamics}

In this section we try to recover some elements of thermodynamics on the generalized ensemble space. We want to recover Gibbs' thermodynamics, which means an equation of state in terms of extensive quantities. Instead of extensive quantities, we are going to use statistical quantities. Note that the idea that all extensive quantities are statistical (i.e. the average during mixing) seems to work. In fact, in statistical mechanics the energy, the number of particles and the volume are averages over microstates. It also seems that intensive quantities do not average during mixing. Temperature, for example, is only defined on equilibria (i.e. of Boltzmann distributions) and the mixture of two equilibria at different temperatures is not an equilibrium. Still, we would need a general proof, which would require the notion of product spaces (i.e. intensive/extensive quantities represent system/subsystem relationship).

\begin{mathSection}
\begin{defn}
	Let $\Ens$ be an ensemble space. Let $F : \Ens \to \mathcal{Q}$ be a statistical property. The \textbf{coarse graining} of $\Ens$ over $F$ is the set $\mathcal{M} \subseteq \Ens$ represented by the ensembles that maximize the entropy for each fixed value of the property. That is, there exists a map $\psi : \mathcal{Q} \to \mathcal{M}$ such that $F(\psi(x)) = x$ and $S(\psi(x)) \geq S(\ens)$ for all $\ens \in \Ens$ such that $F(\ens) = x$. The \textbf{equation of state} is the map $S : \mathcal{Q} \to \mathbb{R}$ defined as $S(x) \mapsto S(\psi(x))$ that returns the entropy given the value of the statistical property.
\end{defn}
\end{mathSection}

The set of ensembles that maximize entropy will be a set of points fully identified by finitely many real numbers. We would expect this to be a manifold, though we still have to understand how the topology of the ensemble space relates to the topology of $\mathcal{M}$.

\begin{mathSection}
\begin{conj}
	Let $F : \Ens \to \mathbb{R}^n$ be a vector of statistical quantities. Then the corresponding coarse graining $\mathcal{M}$ of $\Ens$ is a manifold. The equation of state $S : \mathbb{R}^n \to \mathbb{R}$ returns the entropy as a function of the values of the statistical quantities.
\end{conj}

\begin{remark}
	We need to prove that $\mathcal{M}$ inherits the topology from $\Ens$. Are we be able to recover the topological isolation of phase transitions? The quantities will typically be energy plus other extensive quantities (i.e. volume, number of particles, ...).
\end{remark}
\end{mathSection}

What we can already prove, however, is that the entropy is a strictly concave function of the state variables. Note that, in this setting, the convex combination between state variables does not give a statistical mixture, but rather it gives us the ensemble that maximizes entropy on the averaged constraint. Therefore, the concavity of the entropy on the coarse grained state space is not the same concavity of the entropy on the ensemble space.

\begin{mathSection}
\begin{prop}
	Given a coarse graining $\mathcal{M}$ of $\Ens$, its equation of state is strictly concave. That is, $S(\lambda x + \bar{\lambda} y) \geq \lambda S(x) + \bar{\lambda} S(y)$ and the equality holds if and only if $x=y$.
\end{prop}

\begin{proof}
	Let $x, y \in \mathcal{Q}$ be two possible values for the statistical property, and let $\lambda x + \bar{\lambda} y$ be a convex combination. We have $F(\psi(\lambda x + \bar{\lambda} y)) = \lambda x + \bar{\lambda} y = F(\lambda \psi(x) + \bar{\lambda} \psi(y))$. That is, $\psi(\lambda x + \bar{\lambda} y)$ and $\lambda \psi(x) + \bar{\lambda} \psi(y)$ are two ensembles that share the same value for the statistical property. By definition, the entropy of the first cannot be lower than the entropy of the second. We have
	\begin{equation}
		\begin{aligned}
			S(\lambda x + \bar{\lambda} y) &= S(\psi(\lambda x + \bar{\lambda} y)) \\
			&\geq S(\lambda \psi(x) + \bar{\lambda} \psi(y)) \\
			&\geq \lambda S(\psi(x)) + \bar{\lambda} S(\psi(y)) \\
			&= \lambda S(x) + \bar{\lambda} S(y),
		\end{aligned}
	\end{equation}
	which shows that the equation of state is concave.
	
	To show that the equation of state is strictly concave, suppose that $x=y$. Then on one side $S(\lambda x + \bar{\lambda} y) = S(x)$ and on the other side $\lambda S(x) + \bar{\lambda} S(y) = S(x)$. Therefore $S(\lambda x + \bar{\lambda} y) = \lambda S(x) + \bar{\lambda} S(y)$, which means the inequality holds with the equal. Conversely, suppose that $x\neq y$. Then $\psi(x) \neq \psi(y)$. By the strict concavity of the entropy, we have
	\begin{equation}
		\begin{aligned}
			S(\lambda x + \bar{\lambda} y) &= S(\psi(\lambda x + \bar{\lambda} y)) \\
			&\geq S(\lambda \psi(x) + \bar{\lambda} \psi(y)) \\
			&> \lambda S(\psi(x)) + \bar{\lambda} S(\psi(y)) \\
			&= \lambda S(x) + \bar{\lambda} S(y),
		\end{aligned}
	\end{equation}
	which shows that the equality holds if and only if $x=y$ and therefore the equation of state is strictly concave.
\end{proof}

\begin{remark}
	In the case of thermodynamics, where the statistical property is a vector of statistical values, the equation of state will be concave in all arguments and in all combinations of arguments.
\end{remark}
\end{mathSection}


\section{Orthogonal and separate subspaces}\label{pm-es-subspaceSection}

In this section we recover the notion of subspaces from orthogonality and separateness. Since orthogonality as defined from the entropy coincides with the orthogonality of the inner product for classical and quantum ensemble spaces, orthogonal subspaces as defined here will coincide with those defined by the inner product.

\subsection{Irreflexive symmetric relations and topped $\cap$-structures}

We first present a generic construction, which is sometimes called orthogonality space or orthomodular space, and it sometimes appears in the context of Galois connections. The general idea is that given any irreflexive and symmetric relation, we can define a closure that gives us an orthocomplemented lattice. Since orthogonality and separateness, as defined before, satisfy those properties, they will each give a notion of subspace.

To get an idea, we look at how orthogonality and orthogonal subspaces work in inner product spaces. Let $V$ be an inner product space. If we take a set $U \subseteq V$ of vectors, we can define the set $U^{\perp}$ of all the vectors that are orthogonal to all elements of $U$. This will return the subspace orthogonal to all elements of $U$. We can also define $(U^{\perp})^{\perp}$ as the set of all the vectors that are orthogonal to all elements of $U^{\perp}$. The set $(U^{\perp})^{\perp}$ will contain all the elements of $U$, because they are all orthogonal to all elements of $U^{\perp}$ by definition, but it will also include all the elements in the same subspace. If $U$ were a subspace to begin with, then $U = (U^{\perp})^{\perp}$. Note that we can therefore define subspaces without having a notion of space, just by using the orthogonality relationship, by looking for sets such that $U = (U^{\perp})^{\perp}$.\footnote{This type of construction is similar to some constructions related to Galois connections.}

The construction works for any binary relationship that is symmetric and irreflexive. For example, given two distributions, the fact that they have disjoint support is a symmetric and irreflexive relationship: a distribution does not have disjoint support with respect to itself, and the order of comparison does not matter. We can then construct subspaces based on this relationship, and find sets of functions that are all defined within different regions. In our case, separateness, as defined by the mixing function, and orthogonality, as defined by the entropy, are symmetric and irreflexive.

We start with a generic set $X$ and a symmetric relation (i.e. if $aRb$ then $bRa$). We define the $R$-complement $U^R$ of all elements that are $R$-related to $U$ and study some useful properties. The symmetry of the relation is already enough to recover most of the properties we will need.

\begin{mathSection}
	\begin{defn}
		Let $X$ be a set and $R \subseteq X \times X$ a symmetric relation. Given a subset $U \subseteq X$, we define the \textbf{$R$-complement} to be
		$$ U^{R} = \{ a \in X \, | \, \forall b \in U, aRb  \}. $$
	\end{defn}
	
	\begin{prop}\label{pm-es-rComplProps}
		Let $X$ be a set and $R \subseteq X \times X$ a symmetric relation. Then
		\begin{enumerate}
			\item $U \subseteq V \implies V^{R} \subseteq U^{R}$
			\item $U \subseteq (U^{R})^{R}$
			\item $U^{R} = ((U^{R})^{R})^{R}$
			\item $U^{R} = (V^{R})^{R} \iff (U^{R})^{R} = V^{R}$
			\item $(\bigcup_{i \in I} U_i )^{R} = \bigcap_{i \in I} (U_i)^{R}$
			\item $\emptyset^{R} = X$
		\end{enumerate}
	\end{prop}
	
	\begin{proof}
		1. Suppose $a \in V^{R}$. Then, by definition, $\forall b \in V, aRb$. Since $U \subseteq V$, it is also true that $\forall b \in U, aRb$. Therefore $a \in U^{R}$ by definition. Since $a$ was arbitrary, $V^{R} \subseteq U^{R}$.
		
		2. By expanding the definition of complement, we have $(U^{R})^{R}=\{ a \in X \, | \, \forall b \in U^{R}, aRb \} = \{ a \in X \, | \, \forall b \in \{ c \in X \, | \, \forall d \in U, cRd  \}, aRb \} = \{ a \in X \, | \, \forall b \in X \, s.t. (\forall d \in U, bRd), aRb \}$.
		
		Let $a \in U$ and let $b \in X$ such that $\forall d \in U, bRd$. Since $a \in U$ and $bRd$ for all $b \in U$, we have $bRa$ in particular. Since $R$ is symmetric, $aRb$. Given that $b$ was arbitrary, we conclude that $\forall b \in X \, s.t. (\forall d \in U, bRd), aRb$. Therefore $a \in (U^{R})^{R}$ by definition of complement. Given that $a$ was arbitrary, $U \in (U^{R})^{R}$.
		
		3. We again expand the definition and have $((U^{R})^{R})^{R} = \{ a \in X \, | \, \forall b \in (U^{R})^{R}, aRb \}$.
		
		Let $x \in ((U^{R})^{R})^{R}$. Then $\forall b \in (U^{R})^{R}, xRb$ by definition of the complement. Since by 1. $U \subset (U^{R})^{R}$, we can restrict the previous expression to only the elements of $U$, and therefore $\forall b \in U, xRb$. But this means that $x \in U^{R}$ by definition of the complement. Since $x$ was arbitrary, $((U^{R})^{R})^{R} \subseteq U^{R}$. But by 1., we also have $U^{R} \subseteq ((U^{R})^{R})^{R}$ since $U^{R}$ is just a set onto which we can apply the complement twice . By two-way containment, we have $U^{R} = ((U^{R})^{R})^{R}$.
		
		4. Let $U, V \subseteq X$ such that $U^{R} = (V^{R})^{R}$. Applying the complement on each side, $(U^{R})^{R} = ((V^{R})^{R})^{R}$. By the previous property $((V^{R})^{R})^{R} = V^{R}$ and therefore $(U^{R})^{R} = V^{R}$. Switching $U$ and $V$ proves the other direction.
		
		5. We have:
		\begin{equation*}
			\begin{aligned}
				(\bigcup_{i \in I} U_i )^{R} &= \{ a \in X \, | \, \forall b \in \bigcup_{i \in I} U_i, aRb \} \\
				&= \{ a \in X \, | \, \forall U_i, \forall  b \in U_i, aRb \} \\
				&= \{ a \in X \, | \, \forall U_i, a \in \{ c \in X \, | \,  \forall  b \in U_i, cRb \} \} \\
				&= \bigcap_{i \in I} \{ c \in X \, | \, \forall  b \in U_i, cRb\} \\
				&= \bigcap_{i \in I} (U_i)^{R}
			\end{aligned}
		\end{equation*}
		Note that this property does not rely on the symmetry of $R$.
		
		6. Let $a \in X$. There is no $b \in \emptyset$ such that $aRb$. Therefore $\forall b \in \emptyset, aRb$. This means that $a \in \emptyset^{R}$. Since $a$ was arbitrary, $X = \emptyset^{R}$.
	\end{proof}
\end{mathSection}

The next two properties depend on both the symmetry and the irreflexivity.

\begin{mathSection}
	\begin{prop}\label{pm-es-symirreflproperties}
		Let $X$ be a set and $R \subseteq X \times X$ a symmetric and irreflexive relation. Then
		\begin{enumerate}
			\item $U \cap U^{R} = \emptyset$
			\item $X^{R} = \emptyset$
			\item $(U \cup U^{R})^{R} = \emptyset$
		\end{enumerate}
	\end{prop}
	
	\begin{proof}
		1. Let $a \in U$. Since $R$ is irreflexive,  $aRa$ is false. Therefore it is not true that, for all $b \in U$, $aRb$. This means that $a \notin U^{R}$. Since $a$ was arbitrary, $U \cap U^{R} = \emptyset$.
		
		2. Suppose $a \in X^{R}$. Then for all $b \in X$, $aRb$. In particular, we would have $aRa$, which can't be true since $R$ is irreflexive. Therefore $a \notin X^{R}$ and, since $a$ is arbitrary, $X^{R} = \emptyset$.
		
		3. Suppose $a \in (U \cup U^{R})^{R}$. Then $aRb$ for all $b \in U\cup U^{R}$. Since $aRb$ for all $b \in U$, $a \in U^{R}$. But this would mean $aRa$, which is not possible since $R$ is irreflexive. Therefore $(U \cup U^{R})^{R} = \emptyset$.
	\end{proof}
\end{mathSection}

We now define a notion of $R$-subspace by requiring that a subspace is the $R$-complement of its $R$-complement. We then construct the lattice of subspaces and show it satisfies properties we would expect from a lattice of subspaces.

\begin{mathSection}
	\begin{defn}
		Let $X$ be a set and $R \subseteq X \times X$ an irreflexive symmetric relation. Let $U \subseteq X$. The \textbf{$R$-closure of $U$} is $\langle U \rangle_R = (U^{R})^{R}$. An \textbf{$R$-subspace} of $X$ is a set $U \subseteq X$ such that $U = \langle U \rangle_R$. The \textbf{lattice of $R$-subspaces} is the set $\mathfrak{L} = \{ U \subseteq X \, | \, U = \langle U \rangle_R \}$ ordered by inclusion.
	\end{defn}
	
	\begin{coro}
		The lattice of $R$-subspaces $\mathfrak{L}$ is a topped $\bigcap$-structure on $X$ and therefore is also a complete lattice.
	\end{coro}
	\begin{proof}
		The set $\mathfrak{L}$ is a collection of subsets of $X$. Let $\{U_i\}_{i \in I} \subseteq \mathfrak{L}$ be a non-empty family. Then, using the definition of subspace, and the third and fifth properties of \ref{pm-es-rComplProps}, we have
		\begin{align*}
			\bigcap_{i \in I} U_i &= \bigcap_{i \in I} (U_i^{R})^{R} = (\bigcup_{i \in I} U_i^{R})^{R} \\
			&= (((\bigcup_{i \in I} (U_i^{R})^{R})^{R})^{R} = ((\bigcap_{i \in I} (U_i^{R})^{R})^{R})^{R} \\
			&= ((\bigcap_{i \in I} U_i)^{R})^{R}.
		\end{align*}
		Therefore $\bigcap_{i \in I} U_i \in \mathfrak{L}$. This means $\mathfrak{L}$ is an $\bigcap$-structure. Using the second property of \ref{pm-es-symirreflproperties}, $X = \emptyset^{R} = (X^{R})^{R}$. Therefore $\mathfrak{L}$ is a topped $\bigcap$-structure. This also means that it is a complete lattice.
	\end{proof}
	
	\begin{prop}
		The lattice of $R$-subspaces is orthocomplemented. That is
		\begin{enumerate}
			\item the $R$-complement is a lattice complement: $A \wedge A^{R} = \emptyset$ and $A \vee A^{R} = X$
			\item $(A^{R})^{R} = A$
			\item $A \subseteq B$ implies $B^{R} \subseteq A^{R}$.
		\end{enumerate}
	\end{prop}
	
	\begin{proof}
		Note that 2 is satisfied because $A$ is an $R$-closure and 3 is already proven in \ref{pm-es-rComplProps}.
		For 1, since the lattice is an $\bigcap$-structure, $A \wedge A^{R} = A \cap A^{R}$ and from \ref{pm-es-symirreflproperties} we have $A \cap A^{R} = \emptyset$. Since the lattice is an $\bigcap$-structure, $A \vee A^{R} \supseteq A \cup A^{R}$. Using \ref{pm-es-rComplProps} and \ref{pm-es-symirreflproperties} we have $((A \vee A^{R})^{R})^{R} \supseteq ((A \cup A^{R})^{R})^{R} = (\emptyset)^{R} = X$.
	\end{proof}
	
	\begin{coro}
		The lattice of $R$-subspaces satisfies de Morgan's laws. That is
		\begin{enumerate}
			\item $(A \vee B)^{R} = A^{R} \wedge B^{R}$
			\item $(A \wedge B)^{R} = A^{R} \vee B^{R}$
		\end{enumerate}
	\end{coro}
	
	\begin{proof}
		Every orthocomplemented lattice satisfies de Morgan's laws.\footnote{See, for example, \href{https://en.wikipedia.org/wiki/Complemented_lattice}{Wikipedia}.}
	\end{proof}
	
	\begin{coro}\label{pm-es-subspaceClosure}
		The $R$-closure satisfies the following properties
		\begin{enumerate}
			\item $U \subseteq \langle U \rangle_R$
			\item $U \subseteq V \implies \langle U \rangle_R \subseteq \langle V \rangle_R$
			\item $\langle \langle U \rangle_R \rangle_R = \langle U \rangle_R $
		\end{enumerate}
		and is therefore a closure operation.
	\end{coro}
	
	\begin{proof}
		1. The first property is true by the second property of \ref{pm-es-rComplProps}.
		
		2. Using the first property of \ref{pm-es-rComplProps} we have $U \subseteq V$ implies $V^{R} \subseteq U^{R}$ which in turn implies $(U^{R})^{R} \subseteq (V^{R})^{R}$. Therefore $\langle U \rangle_R \subseteq \langle V \rangle_R$.
		
		3. Using the fifth property of \ref{pm-es-rComplProps} we have $\langle \langle U \rangle_R \rangle_R = (((U^{R})^{R})^{R})^{R} = (U^{R})^{R} = \langle U \rangle_R$.
	\end{proof}
	
	\begin{prop}
		Let $X$ be a set and $R \subseteq X \times X$ a symmetric and irreflexive relation. Then
		\begin{enumerate}
			\item $\langle U \rangle_R$ is the smallest $R$-subspace containing $U$
			\item if $U, V \in \mathfrak{L}$ then $U = V^{R} \iff U^{R} = V$
		\end{enumerate}
	\end{prop}
	
	\begin{proof}
		1. Let $U \subseteq X$ and $V \in \mathfrak{L}$ such that $U \subseteq V$ and $V \subseteq \langle U \rangle_R$. Since $U \subseteq V$, using the second property of \ref{pm-es-subspaceClosure}, $\langle U \rangle_R \subseteq \langle V \rangle_R = V$. Since $V \subseteq \langle U \rangle_R$ and  $\langle U \rangle_R \subseteq  V$, $\langle U \rangle_R = V$. This means that no $R$-subspace that contains $U$ is smaller than $\langle U \rangle_R$.
		
		2. Since $U, V \in \mathfrak{L}$, $U = (U^{R})^{R} = V^{R}$ which, by the fourth property of \ref{pm-es-rComplProps}, implies $U^{R} = (V^{R})^{R} = V$. Switching $U$ and $V$ proves the other direction.
	\end{proof}
\end{mathSection}

We now show that if we start with the mere notion of orthogonality defined from an inner product vector space, we recover the notion of orthogonal components and subspaces. That is, the full structure of subspaces of an inner product space can be fully recovered only from pairwise orthogonality.

\begin{mathSection}
	\begin{prop}
		Let $X$ be an inner product space and $R = \{(a,b) \,|\, \langle a , b\rangle = 0\}$. Then
		\begin{enumerate}
			\item $R$ is an irreflexive and symmetric relation
			\item $U^{R} = U^{\perp}$
			\item $\langle U \rangle_R = \cl_X (\vspan(U))$
		\end{enumerate}
	\end{prop}
	
	\begin{proof}
		1. Given that the inner product is symmetric, so will be $R$. Given that no vector is orthogonal to itself, $R$ is irreflexive.
		
		2. The orthogonal complement is defined as $U^\perp = \{ a \in X \, | \, \forall b \in U, \langle a , b\rangle = 0 \}$. Since $aRb \iff \langle a , b\rangle = 0$, $U^{R} = U^{\perp}$.
		
		3. We have $\langle U \rangle_R = (U^{\perp})^\perp$ which returns the smallest closed subspace that contains $U$.
	\end{proof}
\end{mathSection}

\subsection{Separate and orthogonal subspaces}

Since orthogonality and separateness are irreflexive symmetric operations, they will both give a lattice of subspaces. Here we present a series of conjectures about these subspaces that we have yet to prove.

\begin{mathSection}
\begin{defn}
	Let $\Ens$ be an ensemble space. An $\ortho$-subspace is an R-subspace defined by $\ortho$ and a $\separate$-subspace is an R-subspace defined by $\separate$.
\end{defn}
\end{mathSection}

In the classical case, since both orthogonality and separateness coincide with disjoint support, a subspace consists of all the probability distribution whose support is within a given set $U$.\footnote{Clearly, not all closed sets will give a different subspace. For example, over $\mathbb{R}$, $[0,1]$ and $[0,1]\cup[2,2]$ will correspond to the same subspace. Characterizing that equivalence class is still an open question.}

\begin{mathSection}
\begin{prop}
	Let $\Ens$ be a discrete or continuous classical ensemble space over a sample space $X$. Then $A \subseteq \Ens$ is an $\ortho$-subspace if and only if there exists $U \subseteq X$ such that $A = \{ p \in \Ens \, | \, p(U) = 1\}$.
\end{prop}

\begin{proof}
	Recall that in a classical ensemble space, $p \ortho \lambda$ if and only if $\lambda(\support(p)) = 0$. Let $A \subseteq \Ens$ be an $\ortho$-subspace. Let $U = \bigcup_{p \in A} \support(p)$ be the union of all supports. Then $\lambda \in A^{\ortho}$ (i.e. is orthogonal to all elements of $A$) if and only if $\lambda(U) = 0$. Conversely, $p \in A$ only if it is orthogonal to all elements, which means if and only if $p(U) = 1$. Therefore if $A$ is a subspace, there is a $U \subseteq X$ such that $A = \{ p \in \Ens \, | \, p(U) = 1\}$.
	
	Conversely, let $A = \{ p \in \Ens \, | \, p(U) = 1\}$ for some $U \subseteq X$. Then $A^{\ortho} = \{ p \in \Ens \, | \, p(U) = 0\}$. The orthogonal complement of the orthogonal complement will return $A$, which means $A = (A^{\ortho})^{\ortho}$ is a subspace.
\end{proof}
\end{mathSection}

In the quantum case, the orthogonality coincides with orthogonality in the Hilbert space, and therefore an $\ortho$-subspace coincides with the set of density operators supported by a given subspace.

\begin{mathSection}
\begin{prop}
	Let $\Ens$ be a quantum ensemble space over a Hilbert space $\mathcal{H}$. Then $A \subseteq \Ens$ is an $\ortho$-subspace if and only if there exists a subspace $U \subseteq \mathcal{H}$ with a corresponding projector $\mathbf{1}_{U}$ such that $A = \{ \rho \in \Ens \, | \, \tr[\mathbf{1}_{U} \, \rho] = 1\}$. 
\end{prop}

\begin{proof}
	Recall that to each subspace $U \subseteq \mathcal{H}$ is associated a projector $\mathbf{1}_U$ whose eigenvectors with eigenvalue $1$ span $U$. Given a density operator $\rho \in \Ens$, its support is the subspace $U=\support(\rho)$ spanned by the non-zero eigenvectors. Therefore $\tr[\mathbf{1}_{U} \, \rho] = 1$ and $\tr[\mathbf{1}_{U^{\ortho}} \, \rho] = 0$.

	Also recall that, for a quantum ensemble space, $\rho \ortho \sigma$ if and only if $\tr[ \mathbf{1}_{\support(\rho)} \, \sigma] = 0$. Let $A \subseteq \Ens$ be an $\ortho$-subspace. Let $U = \bigvee_{\rho \in A} \support(\rho)$ be the subspace $U$ spanned by all supports. Then $\sigma \in A^{\ortho}$  (i.e. is orthogonal to all elements of $A$) if and only if $\tr[\mathbf{1}_{U} \, \sigma] = 0$. Conversely, $\rho \in A$ if and only if it is orthogonal to all elements of $A^{\ortho}$, which is the case if and only if $\tr[\mathbf{1}_{U} \, \rho] = 1$. Therefore if $A$ is a subspace, there is a subspace $U \subseteq \mathcal{H}$ such that $A = \{ \rho \in \Ens \, | \, \tr[\mathbf{1}_{U} \, \rho] = 1\}$.
	
	Now let $A = \{ \rho \in \Ens \, | \, \tr[\mathbf{1}_{U} \, \rho] = 1\}$ for some subspace $U \subseteq \mathcal{H}$. The orthogonal complement $A^{\ortho}$ will be the set of all density operators $\sigma \in \Ens$ such that $\tr[\mathbf{1}_{U} \, \sigma] = 0$. The complement of the complement will return $A$, which means $A = (A^{\ortho})^{\ortho}$ is a subspace.
\end{proof}
\end{mathSection}

Note that, in general, the two notions of subspaces, and therefore their lattices, will be different. Note that a feature of classical ensemble spaces is exactly that orthogonality and separateness coincide, which also means the two lattices will coincide.

Another question is whether the lattice of $\ortho$-subspaces has to be necessarily orthomodular, as one requires, for example, in quantum logic. The cut triangle \ref{pm-es-cutTriangle} is a counterexample. The only orthogonality relationships are $\ens[2] \ortho p \ens[3] + \bar{p}\ens[b]$ and $\ens[3] \ortho p \ens[2] + \bar{p}\ens[a]$. This means that the only $\ortho$-subspaces are the bottom $\bot = \emptyset$, $2= \{\ens[2]\}$, $3= \{\ens[3]\}$, $A2= \{p \ens[2] + \bar{p}\ens[a]\}$, $B3= \{p \ens[3] + \bar{p}\ens[b]\}$ and the top $\top = \Ens$. As we can is see in Fig. \ref{pm-es-fig-cutTriangleOrthoSubspaces}, the subspaces form an $M_6$ lattice, which is not orthomodular. In fact, $3 \subseteq B3$ but $3 \vee (3^{\ortho} \wedge B3) = 3 \vee (A2 \wedge B3) = 3 \vee \bot = 3 \neq B3$. The lack of orthomodularity is due to the multiple complements where one is a subset of the other.

\begin{figure}[H]
	\centering
	\begin{tikzpicture}
		%\draw [help lines] (0,0) grid (10,3);
		
		% Cut triangle
		\coordinate (center) at (1.5,1.2);
		\def\angle{90}
		\def\radius{1.5}
		\coordinate (t) at ($(center)+\radius*({cos(0+\angle)},{sin(0+\angle)})$);
		\coordinate (e2) at ($(center)+\radius*({cos(120+\angle)},{sin(120+\angle)})$);
		\coordinate (e3) at ($(center)+\radius*({cos(240+\angle)},{sin(240+\angle)})$);
		\coordinate (a) at ($(e2)!0.5!(t)$);
		\coordinate (b) at ($(e3)!0.5!(t)$);
		\draw[fill=ensembleFill, thick] (a) -- (e2) node[midway,left] {$A2$} -- (e3) -- (b) node[midway,right] {$B3$} -- cycle;
		\filldraw [black] (a) circle (1pt) node[above left] {$\ens[a]$};
		\filldraw [black] (e2) circle (1pt) node[below left] {$\ens[2]$};
		\filldraw [black] (e3) circle (1pt) node[below right] {$\ens[3]$};
		\filldraw [black] (b) circle (1pt) node[above right] {$\ens[b]$};
		\draw [dashed] (a) -- (t) -- (b);

		\node (t) at (6,3) {$\top = \Ens$};
		\node (A2) at (5,2) {$A2$};
		\node (B3) at (7,2) {$B3$};
		\node (X) at (5,1) {$2=\{\ens[2]\}$};
		\node (Y) at (7,1) {$3=\{\ens[3]\}$};
		\node (b) at (6,0) {$\bot = \emptyset$};
		\draw (b) -- (X) -- (A2) -- (t) -- (B3) -- (Y) -- (b);
		\draw[color=red!75!black] (X) -- (B3);
		\draw[color=red!75!black] (Y) -- (A2);
		\draw[color=red!75!black] (t) -- (b);
	\end{tikzpicture}
	%\includegraphics[width=0.7\textwidth]{tempimages/CutTriangleSubspaces.jpg}
	\caption{On the left, the cut triangle. On the right, the lattice of the orthogonal subspaces. The red lines connect the orthogonal complements.}\label{pm-es-fig-cutTriangleOrthoSubspaces}
\end{figure}

This leads to the following:
\begin{conj}
	The lattice of $\ortho$-subspaces is orthomodular if and only if the ensemble space is orthogonally decomposable (i.e. every decomposable ensemble is orthogonally decomposable).
\end{conj}
If this is true, the requirement of orthomodularity is an additional requirement that can be understood physically.

Note, however, that the lattice of $\separate$-subspaces  of the cut triangle is orthomodular, though not commutative. This shows how the two lattices can be very different.

\begin{figure}[H]
	\centering
	\begin{tikzpicture}
		%\draw [help lines] (0,0) grid (10,3);
		
		\node (t) at (4.5,6) {$\top = \Ens$};
		\node (2P) at (0,4.5) {$2P=AB \cup B3$};
		\node (AP) at (3,4.5) {$AP=B3 \cup 23$};
		\node (BP) at (6,4.5) {$BP=23 \cup A2$};
		\node (3P) at (9,4.5) {$3P=A2 \cup AB$};
		\node (B3) at (0,3) {$B3=\{pB+\bar{p}3\}$};
		\node (23) at (3,3) {$23=\{p2+\bar{p}3\}$};
		\node (A2) at (6,3) {$A2=\{pA+\bar{p}2\}$};
		\node (AB) at (9,3) {$AB=\{pA+\bar{p}B\}$};
		\node (3) at (0,1.5) {$3=\{\ens[3]\}$};
		\node (2) at (3,1.5) {$2=\{\ens[2]\}$};
		\node (A) at (6,1.5) {$A=\{\ens[a]\}$};
		\node (B) at (9,1.5) {$B=\{\ens[b]\}$};
		\node (b) at (4.5,0) {$\bot = \emptyset$};
		\draw (b) -- (3) -- (B3) -- (2P) -- (t);
		\draw (b) -- (2) -- (23) -- (AP) -- (t);
		\draw (b) -- (A) -- (A2) -- (BP) -- (t);
		\draw (b) -- (B) -- (AB) -- (3P) -- (t);
		\draw (3) -- (23) -- (BP);
		\draw (2) -- (A2) -- (3P);
		\draw (A) -- (AB) -- (2P);
		\draw (B) -- (B3) -- (AP);
	\end{tikzpicture}
	%\includegraphics[width=0.7\textwidth]{tempimages/CutTriangleSubspaces2.jpg}
	\caption{In this diagram, the lower tier represents the singletons of the extreme points, the middle tier represents each side and the top tier represents the pairs of sides that are the $\separate$-complement of the extreme points. The red lines connect the complements. Note that the lattice is not distributive (e.g. $A\vee (3 \wedge A2) = A \vee \bot = A \neq (A\vee 3) \wedge (A \vee A2) = BP \wedge A2 = A2$). In fact, $\{\bot, 3, A, A2, BP\}$ is an $N_5$ sublattice. }
\end{figure}

\section{Examples}

In this section we plan to collect examples that appear throughout the chapter to exemplify corner cases, wanted or unwanted.

\subsection{Cut triangle}

The cut triangle is an example of a simple classical space with a constraint, which makes it an interesting candidate for testing concepts and definitions on a small deviation from the classical case.

\begin{example}[Cut triangle]\label{pm-es-cutTriangle}
	The cut triangle is the subset of probability measures over three elements where the probability of the first is constrained to be no more than $\frac{1}{2}$.
\end{example}

\begin{figure}[H]
	\centering
	\begin{tikzpicture}		
		%\draw [help lines] (0,0) grid (10,3);
		
		\coordinate (center) at (1.5,1.2);
		\def\angle{90}
		\def\radius{1.5}
		\coordinate (t) at ($(center)+\radius*({cos(0+\angle)},{sin(0+\angle)})$);
		\coordinate (2) at ($(center)+\radius*({cos(120+\angle)},{sin(120+\angle)})$);
		\coordinate (3) at ($(center)+\radius*({cos(240+\angle)},{sin(240+\angle)})$);
		\coordinate (a) at ($(2)!0.5!(t)$);
		\coordinate (b) at ($(3)!0.5!(t)$);
		\coordinate (c) at ($(2)!0.666!(b)$);
		\draw[fill=ensembleFill, thick] (a) -- (2) -- (3) -- (b) -- cycle;
		\filldraw [black] (a) circle (1pt) node[above left] {$\ens[a]$};
		\filldraw [black] (2) circle (1pt) node[below left] {$\ens[2]$};
		\filldraw [black] (3) circle (1pt) node[below right] {$\ens[3]$};
		\filldraw [black] (b) circle (1pt) node[above right] {$\ens[b]$};
		\filldraw [black] (c) circle (1pt) node[below] {$\ens[c]$};
		\draw (a) -- (3);
		\draw (2) -- (b);
	\end{tikzpicture}
	\caption{Cut triangle.}\label{pm-es-fig-cutTriangle}
	%\includegraphics[width=0.4\textwidth]{tempimages/MultipleDecomposition.jpg}
\end{figure}

This ensemble space is the standard two simplex, a triangle, where the top is cut off. This is the space of probability distributions $[p_1, p_2, p_3]$ such that $p_1 \leq \frac{1}{2}$, where entropy is the Shannon entropy. The extreme points of the space are $\ens[2] = [0,1,0]$, $\ens[3] = [0,0,1]$, $\ens[a] = [\frac{1}{2},\frac{1}{2},0]$ and $\ens[b] = [\frac{1}{2},0,\frac{1}{2}]$. Every ensemble that is decomposable is also separately decomposable but not necessarily orthogonally decomposable. Most ensembles, all those in the interior, are also separately multidecomposable. For example, $\ens[c]$ can be expressed as a mixture of $\ens[a]$ and $\ens[3]$, and of $\ens[b]$ and $\ens[2]$.

\subsection{Real interval with right topology}

This example shows how one can have a topological convex space that embeds in a vector space, but its topology is not compatible with the vector space operations. In a topological vector space, the multiplication by $-1$ forces sets that are the mirror image of open sets to be open. This is not guaranteed in a topological convex space.

Note that the topology in the example is $\mathsf{T}_0$ but not $\mathsf{T}_1$. The existence of an entropy function forces the space to be at least Hausdorff, therefore this is not a valid ensemble space.

\begin{example}[Real interval with right topology]
	Let $\Ens$ be the interval $[0,1] \subset \mathbb{R}$ with the topology generated by sets of the form $(r,1]$. Since the topology is not $\mathsf{T}_1$, it cannot be an ensemble space, but the mixing operation is continuous.
\end{example}

To show that the mixing operation is continuous, it suffices to show that the inverse image of sets of the form $(r,1]$ is open. To do that, we show that each point in $+^{-1}((r,1]) = \{(p, a, b) \in [0,1]^3 \, | \, pa+\bar{p}b > r \}$ has an open neighborhood. First, note that sets of the form $(p,1]\times(a,1]\times(b,1]$ and $[0,p)\times(a,1]\times(b,1]$ are open because they are the product of open sets. Next note that if $pa+\bar{p}b>m$, then if $a\geq b$, for any $\lambda>p$, $c>a$ and $d>b$, $\lambda c+\bar{\lambda}d>m$. This is because the convex combination can only increase if the elements increase and/or if the coefficient of the greater element increases. In terms of sets, if $pa+\bar{p}b>m$ and $a \geq b$, $+((p,1],(a,1],(b,1]) \subseteq (m,1]$. Conversely, if $a\leq b$, $+([0,p),(a,1],(b,1]) \subseteq (m,1]$. Now, let $pa+\bar{p}b=m>r$. Then $pa+\bar{p}b>\frac{m+r}{2}$. If $a \geq b$, $+((p,1],(a,1],(b,1]) \subseteq \left(\frac{m+r}{2},1\right]\subseteq (m,1]$. If $a<b$,  $+([0,p),(a,1],(b,1]) \subseteq \left(\frac{m+r}{2},1\right]\subseteq (m,1]$. Therefore, every point in $+^{-1}((r,1])$ has an open neighborhood, and $+^{-1}((r,1])$ is an open set. This means that the mixing operation is continuous.

\section{Lessons learned}

This section summarizes insights from previous attempts that failed. We collect them here for reference.

\subsection{Entropy and convex structure}

\begin{insight}
	The entropy is not uniquely determined by the convex structure.
\end{insight}

When defining the entropy given a space of probability distributions, there is typically an underlying assumption that all ``pure states'' are equivalent. That is, they have the same entropy. For example, in a discrete classical space, the entropy is typically $-\sum_i p_i \log p_i$ which assumes the entropy is zero for all extreme points. This assumption is not tenable in general.

\begin{example}[Mixed gas]
	Let's assume we have a mixture of two gases at a fixed temperature and volume. The state is therefore defined by the variables $n_a$ and $n_b$ that represent the number of molecules of the respective gases. The entropy of each configuration is $\log 1 = 0$. If we have a uniform distribution over $[0,N_a]$ for $n_a$ and $[0,N_b]$ cases of $n_b$, the total number of cases is $(N_a + 1) (N_b + 1)$, and therefore the entropy of the joint state is $\log ((N_a + 1) (N_b + 1))$. Now, suppose we cannot control the number of molecules for each gas, but only the total $n$. Each value of $n$ will not correspond to the same variability of the elements of the ensemble. If we fix $n$ number of molecules, in fact, there will be $n+1$ configurations possible, and therefore the entropy of each case should be $\log (n+1)$.
	
	If we have the space of probability distributions over a single variable that represents the total number of molecules, then we would not be able to know whether these are all indistinguishable or divided into two distinct types. Therefore we would not know which is the correct physical entropy.
\end{example}

Note that in quantum mechanics there is a tacit assumption that all pure states have the same entropy. We could imagine, however, a theory where the pure states, the ones that we can ideally prepare and control, are not all at the same entropy. This would bake into the state space that not all evolutions are reversible. There is no reason to exclude this case, and it may turn out to be what we need for future theories.

\subsection{Continuity of affine combinations}

\begin{insight}
	Mixing cannot be a homeomorphism.
\end{insight}

To answer whether the embedding of the ensemble space in the topology of the vector space is continuous, we wanted to understand whether the inverse of the mixing is continuous. It turns out, it can't be in general because of the extreme points.

\begin{prop}
	Let $\Ens = [0,1]$ and suppose that mixing is a homeomorphism, meaning $+_{p \ens[a]}(\ens[b]) = p\ens[a] + \bar{p}\ens[b]$ is a homeomorphism onto its image. Then $\Ens$ would have to have the discrete topology, which is not possible, since the topology must be second countable.
\end{prop}

\begin{proof}
	Consider $+_{\frac{1}{2}0}([0,1])= [0,\frac{1}{2}]$. Since $[0,1] = \Ens$ is an open set, then $[0,\frac{1}{2}]$ is an open set as $+_{\frac{1}{2}0}([0,1])$ is a homeomorphism. Similarly, $+_{\frac{1}{2}1}([0,1])= [\frac{1}{2},1]$ is an open set. This means that $\left\{\frac{1}{2}\right\} = [0,\frac{1}{2}] \cap [\frac{1}{2},1]$ is an open set. This means that, for every $p \in [0,1]$, $+_{p0}(\{\frac{1}{2}\})= \{\frac{1}{2}(1-p)\}$ is an open set. Similarly, for every $p \in [0,1]$, $+_{p1}(\{\frac{1}{2}\})= \{\frac{1}{2}(1+p)\}$ is an open set. This means that every singleton is open, and the topology is discrete. The topology, then, cannot be second countable and $\Ens$ cannot be an ensemble space.
\end{proof}

Note that, in the above example, $+_{\ens[a]\ens[b]} : [0,1] \to \Ens$ would also fail to be a homeomorphism, though it is not clear how this is related to the failure of continuous embedding in a topological vector space (probably failure of scalar multiplication?).

The continuity of the mixing function means we can stretch open sets and still have open sets, but not, in general, shrink them. In retrospect, this makes sense as the bounds created by the boundaries of the convex set are of a different nature than the bounds of a finite precision measurement.

\subsection{Convergence of points without convergence of fraction}

\begin{insight}
	In a limit, the fraction does not necessarily converge to one.
\end{insight}

Since the fraction tells us how much an ensemble is part of another, an intuitive conjecture would be that if $\ens[a]_i \to \ens[a]$ then $\fraction_{\ens[a]}(\ens[a]_i) \to 1$. That is, the fraction increases as we get closer to the limit. This does not work because one can make a limit over the extreme points.

\begin{example}[Pure state convergence in a quantum space]
	Let $\psi_i$ be a sequence of pure states and $\psi$ a pure state such that $\psi_i \to \psi$ and $\psi_i \neq \psi$ for all $i$. For example, on a Bloch sphere, it would be a sequence of points on the surface that converges to a point on the surface. Then, since they are all pure states, we have $\fraction_{\psi}(\psi_i) = 0$. This means $\psi_i \to \psi$ while $\fraction_{\psi}(\psi_i) \to 0 \neq 1$.
\end{example}

\subsection{Measures and topology}

\begin{insight}\label{pm-es-nonZeroMeasureOnBoundaries}
	Boundaries of open sets can have non-zero measure.
\end{insight}

An earlier attempt when trying to find a compatibility condition between measures and topology was to impose that boundaries of open sets have measure zero. The idea was that we cannot associate a non-zero measure to non-terminating conditions. Though the conceptual idea is sensible, the specific implementation does not work. We can find a counterexample on the real line. 

\begin{prop}
	Let $\mu$ be the Lebesgue measure on the real line $\mathbb{R}$. There exists an open set $U$ such that $\mu(U) \neq \mu(\overline{U})$.
\end{prop}

\begin{proof}
	Let $C$ be the \href{https://en.wikipedia.org/wiki/Smith%E2%80%93Volterra%E2%80%93Cantor_set}{Smith–Volterra–Cantor} set. This is a closed set with no interior for which $\mu(C) = \frac{1}{2}$. Let $U = (0,1) \setminus C$. We have that $U$ is an open set and $\overline{U} = [0,1]$. We have $\mu(\overline{U}) = \mu([0,1]) = 1$ while $\mu(U) = \mu((0,1)) - \mu(C) = \frac{1}{2}$.
\end{proof}

\subsection{Subspaces from convex structure}\label{pm-es-failureConvexSubspace}

\begin{insight}
	The convex structure, by itself, cannot define subspaces.
\end{insight}

Initially, we tried recovering the notion of subspaces purely from the convex structure. We did make some progress in the finite-dimensional case, but ultimately this does not work. We identified two problems.

The original definition was as follows.

\begin{defn}
	Let $\Ens$ be a convex space and $X \subseteq \Ens$ be a subset. We say that $X$ is a \textbf{subspace} of $\Ens$ if it contains all the convex combinations and all the components of its elements. That is, for every $\ens_1, \ens_2, \ens_3 \in \Ens$ and $\lambda \in (0,1)$ such that $p \ens_1 + \bar{p} \ens_2 = \ens_3$ we have:
	\begin{itemize}
		\item $\ens_1, \ens_2 \in X$ implies $\ens_3 \in X$
		\item $\ens_3 \in X$ implies $\ens_1, \ens_2 \in X$.
	\end{itemize}
	The \textbf{convex span} of $X$ is the smallest subspace containing $X$.
\end{defn}

\begin{remark}
	As defined, the convex span of two elements will include all their possible mixtures (i.e. the segment that connects them), all possible decompositions (i.e. all lines that pass through them) plus, recursively, all other mixtures and decompositions that can be reached from those. Physically, the idea is that if we act on some ensembles, then we are also acting on all their components and mixtures. Therefore, the proper definition of subspace does not include just the mixtures, but all possible components and all their possible mixtures.
\end{remark}

\begin{example}[Classical discrete spaces]
	Let $S$ be a set of $n$ possible discrete states and let $\Ens$ be the space of probability distributions over the set $S$ (i.e. $\Ens$ is an $n$-simplex and $S$ its extreme points). A subspace $X$ of $\Ens$ is a convex hull of a subset $U$ of $S$. That is, a subspace of $\Ens$ is the space of probability distributions over a subset of the cases. Geometrically, it is one of the sides (possibly recursively) of the simplex.
	
	To see this, first note that the convex hull $X$ of any subset $U$ of extreme points $S$ is a subspace. In fact, it will contain all convex combinations of $U$, and any element can be decomposed in convex combinations of only $U$. Second, note that only convex hulls of a subset of extreme points can be a subspace. In fact, any element of $\Ens$ can be expressed as a non-trivial convex combination of a set of extreme points $U$. Therefore, if an element is present in a subspace $X$, then $U \subset X$, which means all elements of the convex hull of $U$ are in $X$.
\end{example}

\begin{example}[Finite-dimensional quantum spaces]
	Let $\mathcal{H}$ be an $n$-dimensional Hilbert space and let $\Ens$ be the space of density matrices (i.e. positive semi-definite self-adjoint operators with trace one). A subspace $X$ of $\Ens$ is the space of density matrices of a subspace $U$ of $\mathcal{H}$. That is, a subspace of $\Ens$ is the space of mixed states over a subspace of pure states.
	
	To see this, first note that the space of density matrices $X$ of a subspace $U$ of $\mathcal{H}$ is a subspace of $\Ens$. In fact, $X$ will contain all convex combinations of its elements. Moreover, any element $x \in X$ can be decomposed in a convex combination of pure states of only $U$. Therefore any convex decomposition of $x$ has all its elements in $X$. Second, note that only the space of density matrices $X$ of a subspace $U$ of $\mathcal{H}$ is a subspace of $\Ens$. In fact, any element $x$ of $\Ens$ can be expressed as a non-trivial convex combination of orthogonal pure states, its eigenstates. These elements will span a subspace $U$ of $\mathcal{H}$. From those elements, we can construct an equal mixture which represents the maximally mixed states and, mathematically, is the identity operator $I/m$ divided by the number of elements $m \leq n$ of $U$. The equal mixture of any orthogonal basis of $U$ will also give the maximally mixed state. Therefore, given an element $x$, any subspace that contains $x$ will also contain a basis of $U$, the maximally mixed state $I/n$, all possible basis of $U$, which means all the pure states, and finally all convex combinations of the pure states, which means all possible density matrices, all possible mixed states.
\end{example}

The definition works in these cases, which is why it looked promising, but not in general.

\begin{insight}
	Rate of convergence cannot be changed by a convex combination.
\end{insight}

\begin{prop}
	Let $\Ens$ be the space of probability measures over $[0,1]$. Let $\ens[a]$ be the uniform distribution and let $\ens[b]$ be a distribution whose density goes to zero at the endpoints. Then $\ens[b]$ is a component of $\ens[a]$ but not vice-versa. Therefore the convex span of $\ens[a]$ is the whole $\Ens$ while the convex span of $\ens[b]$ is not the whole $\Ens$.
\end{prop}

\begin{proof}
	Consider the two probability densities $\rho_a$ and $\rho_b$ associated with the ensembles. The density $\rho_b$ has a supremum $\sup \rho_b$ that is greater than $1$, which means $p = \frac{1}{\sup \rho_b}$ is smaller than one. The function $\rho_c = \frac{1}{\bar{p}} \rho_a - \frac{p}{\bar{p}}\rho_b$ will be a non-negative function that integrates to one. The same procedure could be applied for any probability density, which means every probability measure over $[0,1]$ is a component of the uniform distribution. Therefore the convex span of $\ens[a]$ is the whole $\Ens$.
	
	Conversely, consider a possible convex combination $p \rho_a + \bar{p} \rho_c = \rho_b$. Since $\rho_b$ converges to zero at the endpoints, for each $p$ there will be a neighborhood of $0$ for which $p\rho_a$ is greater than $\rho_b$. Therefore $\rho_c$ cannot be a non-negative function. This means that $\ens[a]$ is not a component of $\ens[b]$ and the convex span of $\ens[b]$ is not the whole $\Ens$.
\end{proof}

This means that this notion of subspace does not simply return all the functions with the same support, but the function with the same support and a particular class of convergence on the boundaries, and possibly at interior points.

One key problem is that it cannot be generalized to the classical continuous case, as the rate of convergence cannot be changed by finite convex combinations. Even if that problem were fixed, there would be nothing to determine the dimensionality of $U$, and we could create convex maps that ``stretch'' the space. This led to the following:

\begin{insight}
	Dimension/count of states cannot be determined by the convex structure.
\end{insight}

Even if the previous problem is solved, the ``size'' of the subspace cannot be determined with a convex structure alone. In the finite-dimensional case this works, but in the infinite-dimensional case it does not.

\begin{prop}
	Let $X=M_1([0,1])$ and $Y=M_1([0,2])$ be the spaces of probability measures on $[0,1]$ and $[0,2]$ respectively. The function $f : [0,1] \to [0,2]$ such that $f(x) = 2x$ is a diffeomorphism and induces a bijective continuous affine map between $M_1([0,1])$ and $M_1([0,2])$. However, the entropy is not conserved through the map.
\end{prop}

\begin{proof}
	In terms of the probability measures, the map simply acts on the transformed sets. That is, $\mu_Y(A) = \mu_X(f^{-1}(A))$. This, for example, maps the uniform distribution over $[0,1]$ to the uniform distribution over $[0,2]$. Therefore $A$ and $B$ are isomorphic as convex spaces. However, the entropy over a uniform distribution changes with the range, therefore the entropy is not conserved through the map.
\end{proof}

This means that the convex structure and the topology are not enough to determine the size of the space. This means that, without information about the entropy, we are not going to be able to tell that we spread ensembles over double the distance.


\subsection{Spectrum from subspaces}

\begin{insight}
	The lattice of subspaces is not enough to recover the points.
\end{insight}

Originally, we intended to recover the spectrum of an operator (i.e. the possible values over which the distribution is defined) by taking the limit of subspaces. For example, for position a possible value $x$ would be the limit of the sequence of subspaces of distributions with support $[x-\epsilon, x+\epsilon]$. Formally, the construction was similar to \href{https://en.wikipedia.org/wiki/Stone%27s_representation_theorem_for_Boolean_algebras}{Stone's representation theorem for Boolean algebras}, and the points were the ultrafilters. However, this does not work as the ultrafilters are ``too fine.'' When reconstructing a real line, for example, the limit approaching from below and the limit approaching from above would be two separate points.

The core of the problem is that the lattice of subspaces recovers the Boolean algebra of the regular open sets. However, the lattice of the regular open sets is not enough to reconstruct the points of the space.

\begin{prop}[Regular open sets are not enough]\label{pm-es-regularOpenSetsNotEnough}
	Let $X$ and $Y$ be two topological spaces such that the respective lattices of regular open sets $R_X$ and $R_Y$ are isomorphic as Boolean algebras. It does not follow that $X$ and $Y$ are homeomorphic as topological spaces.
\end{prop}

\begin{proof}
	Let $X=S^1$ be a circle with the standard topology, $Y=\mathbb{R}$ be the real line with the standard topology and $Z=\mathbb{R}\setminus \{0\}$ be the real line without the origin. Note that the circle is the one-point compactification of the real line, and therefore $Y=X\setminus \{\infty\}$. This forms a chain of embedding $Z \to Y \to X$.
	
	Suppose $U_X$ is a regular open set in $X$. Then $U_Y = U_X \setminus \{\infty\}$ is an open set in $Y$. We also have $\overline{U_Y} = \overline{U_X} \setminus \{\infty\}$ and $\interior(\overline{U_Y}) = \interior(\overline{U_X}) \setminus \{\infty\}$ where the closure and interior are taken in the respective spaces. Therefore $U_Y$ is a regular open set. Also, let $U_X$ and $V_X$ be two distinct open sets of $X$. Then, since every open set that contains $\{\infty\}$ also contains an open neighborhood of $\{\infty\}$, the corresponding $U_Y$ and $V_Y$ will also be two distinct open sets. Then $\iota(U_X) = U_X \setminus \{\infty\}$ is an injection between the regular open sets of $X$ and $Y$. Now let $U_Y$ be a regular open set on $Y$. Let $U_X = \interior(\overline{U_Y \cup \{ \infty \}})$ where the  closure and interior are taken in $X$. Then $U_X$ is a regular open set of $X$ because it is the interior of a closure. Also note that $\interior(\overline{\iota(U_X) \cup \{ \infty \}}) = \interior(\overline{U_X \setminus \{\infty\} \cup \{ \infty \}}) = \interior(\overline{U_X}) = U_X$. Therefore the map $\iota$ is a bijection between the regular open sets of $X$ and $Y$.
	
	Note that regular open sets form a Boolean algebra where the complement is the exterior, the join is the interior of the closure of the union and the meet is the intersection. Note that if $U_X \subset V_X$, then $\iota(U_X) \subset \iota(V_X)$, therefore $\iota$ preserves the joins and the meets. Now let $V_X = \exterior(U_X)$ and consider $\iota(V_X)$. This will contain all the exterior points of $U_X$ minus $\{\infty\}$. But these are also the exterior points of $\iota(U_X)$. Therefore $\exterior(\iota(U_X)) = \iota(\exterior(U_X))$. The bijection is an isomorphism of Boolean algebras, and the corresponding algebras of regular open sets are isomorphic. The same reasoning applies when comparing $Y$ to $Z$.
	
	Now, note that $\{\infty\}$ is a closed set, but not a regular closed set. Therefore we are going to be able to construct a similar argument for the lattice of regular closed sets
\end{proof}

Since the construction in terms of subspaces recovered the algebra of regular open (or closed) sets, and wanted to recover the points as limits, the above proposition implies that this is not possible. We would not be able to know, for example, whether the points form a circle or a real line. Even if we knew that they form a real line, and knew what half-lines we have, we would not know how to stitch them back together, as we could do it with $\{0\}$ or with $\{\infty\}$. We would need to know, at least, which intervals correspond to a finite interval (e.g. a finite entropy), which requires more structure.

\ifdraft

\titledbreak{Open problems and old attempts}

There are still several open problems, with half-finished or failed attempts. Here we collect some of the ideas and trials, which likely include wrong results. They should mainly serve as inspiration for future attempts.

\begin{openproblem}{Affine sets and affine hulls (revisited)}{pm-es-affineHullProblem}{Ensemble spaces, Convex spaces, Affine spaces}
	In the context of ensemble spaces, we have defined closures in terms of convex combinations. It may be useful to also define closures in terms of affine combinations. It needs to be understood whether this is useful, and how to make sure that these closures work for infinite dimensional spaces.
\end{openproblem}

The following notes represent an early attempt. It needs to be understood whether this is at all needed and, in that case, revised and finalized. We used the term flat in this notes, but if may be better to first define an affine set as one closed under affine combinations and then the affine hull. This would be more consistent with the previous definitions and some of the literature.

In some cases, it may useful to define sets closed under affine combinations instead of convex combinations. That is, if $\ens[a] = p \ens[b] + \bar{p} \ens[c]$ and two of the ensembles are in the set, then the other one is in the set as well. This allows us to get all the ensembles in the same hyperplane. We call this a flat. In the triangle example, the only flats available are the single vertexes, each side and the whole triangle. For the Bloch ball, if we took two elements, we would get the whole segment that connects them and extends to the surface. But if we get three points, we get a circle, which is not a simplex. This affine closure, essentially, allows us to remove ``ensembles from each other'' as much as possible, so that we can get to their most distinct, most separate, form.

It still needs to be understood what the best definition for flat is, its relationship with the topological closure, its corresponding hull and so on.

\begin{defn}
	A \textbf{simplex} is a subset $U \subset \mathbb{R}^n$ that is the hull of finitely many affinely independent points. That is, $U = \hull(\{x_i\}_{i=0}^{n})$ such that $\{x_i - x_0\}_{i=1}^{n}$ are linearly independent.
\end{defn}

\begin{defn}
	A \textbf{flat} $A \subseteq \Ens$ is a closed convex subset that contains all lines between all elements. That is, for any $\ens[a], \ens[b] \in A$, $A$ also contains the line that contains $\ens[a]$ and $\ens[b]$. Given a set $U \subseteq \Ens$, the \textbf{flat closure} of $U$ is the smallest flat that contains $U$. A flat is \textbf{finite} if it can be generated by a finite number of elements. An $n$-flat is a flat that must be generated by a set with at least $n$ elements.
\end{defn}

\begin{figure}[H]
	\centering
	\includegraphics[width=0.4\textwidth]{tempimages/FlatVsConvex.jpg}
	\caption{Flat vs convex set. The ball is the ensemble space. The triangle represents a convex subset. The disc containing the triangle is the flat closure of the triangle. The plane is the affine closure of the triangle.}
\end{figure}


\begin{prop}
	A flat includes all possible affine combinations of elements of $\Ens$ that are contained in $\Ens$. The flat closure of $U \subseteq \Ens$ is the intersection of $\Ens$ with the affine closure of $U$ in the embedding vector space.
\end{prop}

\begin{proof}
	Let $A \subseteq \Ens$ be a flat and let $V$ be the real vector space that embeds $\Ens$. A line between two elements $\ens[a], \ens[b] \in A$ can be extended in $V$ and can be written as $\ens[a] + x \left(\ens[b] - \ens[a]\right)$ with $x \in \mathbb{R}$. Therefore any ensemble that can be written as an affine combination of two ensembles in $A$ is an element of the flat. Recursively, this means that any ensemble that can be written as an affine combination of finitely many ensembles in $A$ is also in the flat. Moreover, since the flat is a closed set, it will also include all its limits. Therefore every affine combination of $A$ that is also an element of $\Ens$ is in $A$, which means $A$ is the intersection of an affine subspace of the embedding vector space and the ensemble space.
	
	Now let $U \subseteq \Ens$ be a set of ensembles. The affine closure of $U$ will be an affine subspace of the embedding vector space. The flat closure will be the intersection of the affine closure with $\Ens$. Since the affine closure is the smallest closed affine subspace that contains $U$, its intersection with $\Ens$ will be the smallest flat that contains $U$, which is the flat closure of $U$.
\end{proof}

\begin{conj}
	An $n$-flat is a simplex if and only if its $3$-flats are simplexes (i.e. triangles).
\end{conj}

\begin{remark}
	In classical discrete ensemble spaces, any flat is a simplex. In classical continuous ensemble spaces, infinite dimensional flats will depend on what limits are allowed. In a quantum ensemble space, if we take three ensembles inside a Bloch ball, the corresponding flat will be a circle. If we take three orthogonal pure states, however, the corresponding flat will be a simplex.
\end{remark}


\section{Reducibility}

Another open question is the right characterization of classical spaces. Part of the problem is isolating the different parts that may lead to different features that one may consider classical.

For example, finite discrete classical spaces are simplexes such that all extreme points are orthogonal to each other and have zero entropy. One may have a simplex where all extreme points are orthogonal, though the extreme points have different entropy. All features of a classical theory are there, expect that the entropy function has to be corrected to $S(\ens) = I(p_i) + \sum_i p_i S(x_i)$. A weaker case is when the ensemble space may be a simplex, but the extreme points are not orthogonal.

The definition will need to work for the classical continuous case. Given that classical phase space is not, in general compact, the space of probability distribution is not a Choquet simplex. Moreover, even if we restrict to ourselves to the compact case, the Dirac measures cannot be in the ensemble space, as their entropy is minus infinity. 

Here, we focus on the intermediate property: the ensemble can be orthogonally decomposed, but the entropy of the extreme points is not necessarily uniform. 




\begin{defn}
	An ensemble space $\Ens$ is \textbf{reducible} if every decomposable ensemble can be decomposed into ensembles with no common components and ensembles that have no common component are mutually exclusive. That is, it is separately decomposable and separateness implies orthogonality (i.e. $\ens_1 \separate \ens_2$ implies $\ens_1 \ortho \ens_2$).
\end{defn}

\begin{justification}
	The idea of reducibility is that we can take ensembles and decompose them into parts that are mutually exclusive. This is the basic expectation in the classical case.
	
	If we assume reducibility, then, if an ensemble is a mixture of other ensembles, we must be able to express it as a mixture of separate ensembles. This justifies separate decomposability.
	
	Given that $\ens_1 \ortho \ens_2$ implies $\ens_1 \separate \ens_2$, reductibility add the opposite implication. Therefore it excludes the case where two ensembles are orthogonal but not separate. This case describes two ensembles that have elements in common, but there is no ensemble corresponding to those common elements. That is, we cannot refine the ensembles into three separate ones: one with only elements of the first, one with only elements of the second and one with elements of both. In other words, we cannot reduce the coarser description of the system into finer separate descriptions. In the excluded case, then, the coarser description is irreducible into finer ensembles. This justifies that, under reducibility, separate ensemble must be orthogonal.
\end{justification}

\begin{prop}
	Continuous and discrete classical ensemble spaces are reducible.
\end{prop}

\begin{proof}
	As we saw before, classical ensemble spaces are separately decomposable. We also saw that both separateness and orthogonal both correspond to disjoint support. Therefore separateness implies orthogonality.
\end{proof}

What we need to prove is that every reducible ensemble space can be represented by a subspace of probability measures. The strategy would be to prove the following series of conjectures. The lattice of $\ortho$-subspaces $\mathfrak{L}^{\ortho}$ forms a Boolean algebra. Using Stone's representation theorem for Boolean algebras, there is a set $X$ and $\mathfrak{L}^{\ortho}$ correspond to the clopen sets, and orthogonal subspaces correspond to disjoint sets. Set functions over clopen sets can be extended to the Borel algebra of $X$. Since the state capacity is additive over orthogonal sets, it is an additive set function when defined to the Borel algebra of $X$ and, therefore, it is a measure. Similarly, one finds that the fraction capacity becomes additive, and, given its other properties, it is a probability measure over $X$. Additionally, since every ensemble has a finite entropy and the state capacity gives us the supremum of the exponential of the entropy, the fraction capacity for a set of measure zero (i.e. minus infinity entropy) must be zero (i.e. no component can exist). 

\begin{conj}
	Every reducible ensemble space can be represented by a space of probability measures. That is, given a reducible ensemble space $\Ens$ there is a topological space $X$ such that each $\ens \in \Ens$ is represented by a unique probability measure $p_{\ens}$ over $X$ that is absolutely continuous with a measure $\mu$ over $X$.
\end{conj}

\section{Classical contexts}

Classical contexts represent subsets of ensemble spaces that can be understood as classical regions. There is again the difference of just considering the convex structure (i.e. they form a simplex) or also add the entropic structure (i.e. separate ensembles are orthogonal). In the attempt below, we chose to only use the convex structure. These are still rough ideas that may need to be refined.

The basic idea is that the hull of a set of orthogonal states in quantum mechanics forms a simplex, so we can understand projection measurements as moving the system to one of those states. Ideally, a classical context should be a subset of an ensemble space that is a classical ensemble space. An additional problem is the multiplicity of extreme points for the same measurement outcome. In that case, the set of post-measurement ensembles are not a simplex. It may be that the correct definition for a classical context would be the domain of a map to simplex that preserves orthogonality.


\subsection{Sets decomposability}

Since contexts set can be a subset of, for example, a quantum ensemble space and this property must work recursively, the first step is to restring the notion of decomposability to sets.

\begin{defn}
	Let $C$ be a convex set. The set is \textbf{separately/orthogonally decomposable} if every element of $C$ that is decomposable in $C$ is also separately/orthogonally decomposable in $C$. The set is \textbf{monodecomposable} if every element in $C$ that is decomposable in $C$ is monodecomposable in $C$. The set is \textbf{separately/orthogonally monodecomposable} if it is both separately/orthogonally decomposable and monodecomposable.
\end{defn}

\begin{defn}
	Let $C$ be a convex set. We say that \textbf{mixtures preserve separateness in $C$} if $\ens \separate \ens[a]$ and $\ens \separate \ens[b]$ implies $\ens \separate p \ens[a] + \bar{p} \ens[b]$ for all $p \in [0,1]$.
\end{defn}

\begin{prop}
	A convex set $C$ is monodecomposable in $C$ if and only if mixtures preserve separateness in $C$.
\end{prop}

\begin{figure}[H]
	\centering
	\includegraphics[width=0.4\textwidth]{tempimages/SeparableButNotFlat.jpg}
\end{figure}

\begin{remark}
	A convex set that is separately decomposable is not necessarily a flat (e.g. triangle within a sphere). A flat is not necessarily separately decomposable (e.g. open segment as a whole is a flat, but is not separately decomposable).
\end{remark}

We can now define a classical probability context as a flat that does not allow multiple decompositions. The definition below may still need to be refined. In particular, we may want need to include the requirement of orthogonality during the decomposition for an actual classical context as classical probability implicitly requires that the cases are mutually exclusive. We would need a different name when the requirement is only on the convex structure, like below.

	\begin{defn}
		Let $\Ens$ be an ensemble space. A \textbf{classical probability context} is a flat $C \subseteq \Ens$ where each decomposable element in $C$ is separately decomposable in $C$ but not separately multidecomposable in $C$.
	\end{defn}

The idea should be that we can use the lack of multidecomposability to split classical contexts recursively.

	\begin{prop}
		A flat $C \subseteq \Ens$ is a classical probability context if and only if every decomposable element is separately decomposable and mixtures preserve separateness in $C$. That is, if $\ens \separate \ens[a]$ and $\ens \separate \ens[b]$, then $\ens \separate p\ens[a] + \bar{p} \ens[b]$ in $C$ for all $\ens, \ens[a], \ens[b] \in C$ and $p \in [0,1]$.
	\end{prop}
	
	\begin{proof}
		Suppose mixtures do not preserve separateness. Then we can find $\ens,\ens[a],\ens[b], \ens[c] \in \Ens$ such that $\ens \separate \ens[a]$, $\ens \separate \ens[b]$ and $\ens \nseparate \ens[c] = p \ens[a] + \bar{p} \ens[b]$ for some $p \in (0,1)$. Since $\ens \nseparate \ens[c]$, we can find $\ens[d], \ens[f], \ens[g]$ such that $\ens[c] = \lambda \ens[d] + \bar{\lambda} \ens[f]$ and $\ens = \mu \ens[d] + \bar{\mu} \ens[g]$.  Since separateness extends to all mixtures (\ref{pm-es-separateExtendsMixtures}) and $\ens[a] \separate \ens[e]$, $\ens[a] \separate \ens[d]$ and, similarly, $\ens[b] \separate \ens[d]$, which means that $\ens[c]$ is separately multidecomposable.
		
		Now suppose mixture do preserve separateness, and let $\ens = p \ens[a]_1 + \bar{p} \ens[a]_2 = \lambda \ens[b]_1 +\bar{\lambda} \ens[b]_2$. Since $\ens[a]_1$ is a component of $\ens$, then $\ens \nseparate \ens[a]_1$. Since $\ens$ is a mixture of $\ens[b]_1$ and $\ens[b]_2$ and mixtures preserve separateness, then either $\ens[a]_1 \nseparate \ens[b]_1$ or $\ens[a]_1 \nseparate \ens[b]_2$. Similarly, $\ens[a]_2 \nseparate \ens[b]_1$ or $\ens[a]_2 \nseparate \ens[b]_2$. Therefore $\ens$ is not separately multidecomposable.
	\end{proof}
	
	\begin{conj}
		A convex subset $U \subseteq \Ens$ is a classical probability context if and only if all finite flats are simplexes.
	\end{conj}
	
	\begin{prop}
		Let $C \subseteq \Ens$ be a classical probability context. Let $U \subseteq C$ be a set of ensembles. Then $A=U^\separate=\{ \ens[a] \in C \, | \, \forall \ens \in U, \ens[a] \separate \ens \}$ and $B=(U^\separate)^\separate=\{ \ens[b] \in C \, | \, \forall \ens[a] \in A, \ens[b] \separate \ens[a] \}$ are two probability contexts such that $C = \hull(A \cup B)$.
	\end{prop}
	
	\begin{proof}
		First we show that $C$ contains only three types of ensembles: those that are limits of convex mixtures of $A$, those that are limits of convex mixtures of $B$ and those that are limits of convex mixtures of both $A$ and $B$. Any ensemble in $C$ is one of these three types. For $\ens[c] \in C$ to not be a mixture of $A$ or $B$, then $\ens[c]$ cannot be in either $A$ or $B$. This means that it must be separate from both $A$ and $B$. But $B$ contains all the elements that are separate from $A$, which is a contradiction. Since separate multidecomposition is forbidden, an ensemble $\ens$ cannot be written both as a convex combination of $A$ and as a convex combination with an element of $B$. This would yield two decompositions in which one component, the one chosen from $B$, is separate from all the components of the other. This means that an element of $C$ is either a mixture of $A$, a mixture of $B$ or a mixture of both $A$ and $B$.
		
		Now we show that both $A$ and $B$ are convex sets. Since a mixture of $A$ can only be expressed as a mixture of components of $A$, it is separate from all elements of $B$. Therefore $A$ contains all its mixtures. With the same logic, $B$ will contain all its mixtures. The argument works for infinite convex combinations as well. If $\ens = \sum_{i=1}^{\infty} p_i \ens[a]_i$ with $p_i \in [0,1]$ and $\sum_{i=1}^{\infty} p_i = 1$, it can be understood as the limit of the series $\frac{p_1}{P_n} \ens[a]_1 + \frac{\bar{p}_1}{P_n} \sum_{i=2}^{n} p_i \ens[a]_i$ where $P_n = \sum_{i=1}^{n} p_i$. Therefore
		\begin{equation}
			\begin{aligned}
				\ens &= \sum_{i=1}^{\infty} p_i \ens[a]_i = \lim\limits_{n \to \infty}  \sum_{i=1}^{n} \frac{p_i}{P_n} \ens[a]_i = \lim\limits_{n \to \infty} \frac{p_1}{P_n} \ens[a]_1 + \lim\limits_{n \to \infty}\frac{\bar{p}_1}{P_n} \sum_{i=2}^{n} \frac{p_i}{\bar{p}_1} \ens[a]_i = p_1 \ens[a]_1 + \bar{p}_1 \sum_{i=2}^{\infty} \frac{p_i}{\bar{p}_1} \ens[a]_i \\
				&= p_1 \ens[a]_1 + \bar{p}_1 \hat{\ens[a]}_1
			\end{aligned}
		\end{equation}
		where $\hat{\ens[a]}_1 = \sum_{i=2}^{\infty} \frac{p_i}{\bar{p}_1} \ens[a]_i$. Since $\ens$ and $\ens[a]_1$ are elements of the ensemble space, the series converges to $\hat{\ens[a]}_1$, which is in the hull of $A$. It will also be an element of $A$ because multidecompositions are forbidden.
		
		To see that $C$ is the hull of $A$ and $B$, note that all the elements in $C$ that are not already in $A$ or $B$ are the mixtures of $A$ and $B$. These are exactly added when taking the hull of $A \cup B$.
		
		Now we show that $A$ and $B$ are classical probability contexts. First we have to show that they are flats. Let $L$ be the line that connects two elements $\ens[a]_1, \ens[a]_2 \in A$. Take $\ens[a]_3 \in L$. If it is a mixture of $\ens[a]_1$ and $\ens[a]_2$ then it is an element of $A$. If $\ens[a]_1$ is a mixture of $\ens[a]_2$ and $\ens[a]_3$, since $\ens[a]_1$ cannot have a common component with $B$, and $\ens[a]_3$ is a component of $\ens[a]_1$, $\ens[a]_3$ cannot have components in $B$ as well. Therefore $\ens[a]_3$ must be a mixture of elements of $A$. Similarly if $\ens[a]_2$ is a mixture of $\ens[a]_1$ and $\ens[a]_3$. Therefore $A$ is a flat. Similarly, $B$ is a flat.
		
		Now we show that $A$, and by symmetry $B$, is a classical probability context. We have seen that $A$ is a flat. If an element of $A$ is decomposable in $A$ it is also decomposable in $C$ and is therefore separately decomposable in $C$. Because multidecomposability in $C$ is not allowed, it must be separately decomposable into elements of $A$. Lastly, if multidecomposability were allowed in $A$, it would also be allowed in $C$. Therefore it is not allowed in $A$. This means that $A$ is a classical probability context.
	\end{proof}
	
	\begin{figure}[H]
		\centering
		\includegraphics[width=0.25\textwidth]{tempimages/CounterexampleProbabilityContexts.jpg}
		\caption{TODO: make the final picture not go through the center, change the label for the points to $\ens_1$ and $\ens_2$}
	\end{figure}
	
	\begin{remark}
		Note that if multiple separate decompositions are not ruled out, the sub-contexts will not include all convex combinations. Take a disk as a convex space. Suppose $U$ is made of two points $\ens_1$ and $\ens_2$. All other points on the surface are separate from both and therefore belong to $A$. Now consider the convex combinations of $\ens_1$ and $\ens_2$. These are not separate from $U$ but they are also not separate from all the other elements on the surface. Therefore they are neither in $A$ nor $B$. Moreover, any other point in the interior can be seen as a convex combination of $\ens_1$ and another element of the surface that is not $\ens_2$. Therefore no point in the interior is in either $A$ or $B$. Thus $A$ and $B$ are not necessarily convex sets if $C$ is not a classical probability context.
	\end{remark}
	
	\begin{figure}[H]
		\centering
		\includegraphics[width=0.75\textwidth]{tempimages/ContextClosureNotContext.jpg}
	\end{figure}
	
	\begin{remark}
		While a classical probability context can be decomposed into classical probability contexts whose closure is the original context, the converse is not true. That is, given two classical probability contexts, their convex or flat closure is not in general a classical probability context. Take a disk (the ensemble space) and two lines connecting three points (the two probability contexts). The convex closure is the triangle, which is not a flat as it misses some affine combinations. The flat closure is the disk, which is not a probability context.
	\end{remark}
	
	\begin{conj}
		Let $A, B \subseteq \Ens$ be two classical probability contexts. Then their flat closure and convex closure coincide if and only if these closures are a classical probability context.
	\end{conj}
	
	\begin{conj}
		The defining property of a classical probability context is exactly that property that allows $A$ and $B$ so constructed to be convex sets whose closure is $C$.
	\end{conj}

\begin{prop}
	Let $C \subseteq \Ens$ be a classical probability context. The lattice of $\separate$-subspaces $\mathfrak{L}$ is distributive and is therefore a Boolean algebra.
\end{prop}

\begin{proof}
	Let $X,Y,Z \in \mathfrak{L}$ be three $\separate$-subspaces. Let $\ens \in X \wedge (Y \vee Z)$, then $\ens \in X$ and $\ens \in Y \vee Z$. This also means that all components of $\ens$ are in both $X$ and $Y \vee Z$ since the components of $\ens$ are not separate from $\ens$ and, given the absence of multi-decomposability, are separate from all elements that are separate from $\ens$. Since $Y \vee Z = \hull(Y \cup Z)$, we can decompose $\ens$ into components of $Y$ and $Z$. But these must be in $X$, and therefore $\ens \in \hull((X \wedge Y) \cup (X \wedge Z) ) = (X \wedge Y) \vee (X \wedge Z) )$. Therefore $X \wedge (Y \vee Z) \subseteq (X \wedge Y) \vee (X \wedge Z)$. Now let $\ens \in (X \wedge Y) \vee (X \wedge Z)$, then $\ens \in \hull((X \wedge Y) \cup (X \wedge Z) )$. Therefore $\ens$ is the mixture of ensembles that are in $X$, which means that $\ens \in X$. Since $\ens$ is the mixture of elements that are also in $Y$ and $Z$, $\ens \in \hull(Y \cup Z) = Y \vee Z$. This means $\ens \in X \wedge (Y \vee Z)$. Therefore $(X \wedge Y) \vee (X \wedge Z) \subseteq X \wedge (Y \vee Z)$. Which means $X \wedge (Y \vee Z) = (X \wedge Y) \vee (X \wedge Z)$.
	
	Now let $\ens \in X \vee (Y \wedge Z)$, then $\ens \in \hull(X \cup (Y \wedge Z) )$. That is, $\ens$ is a mixture of elements of $X$ and of elements that are in both $Y$ and $Z$. Which means $\ens \in \hull(X \cup Y)$ and $\ens \in \hull(X \cup Z)$. Therefore $\ens \in \hull(X \cup Y) \cap \hull(X \cup Z) = (X \vee Y) \wedge (X \vee Z)$, which means $X \vee (Y \wedge Z) \subseteq (X \vee Z) \vee (Y \wedge Z)$. Conversely, let $\ens \in (X \vee Y) \wedge (X \vee Z) = \hull(X \cup Y) \cap \hull(X \cup Z)$. Then $\ens$ can be expressed as a mixture of elements of $X$ and $Y$ and also of elements of $X$ and $Z$. Now consider the elements of $Y$ in the first decomposition. Since multiple decomposition is not allowed, they cannot be separate from both $X$ and $Z$, end therefore must be in their hull. Similarly, the elements of $Z$ in the second decomposition must be in the hull of $X$ and $Y$. Therefore $\ens \in \hull(X \cup (Y \wedge Z) ) = X \vee (Y \wedge Z)$. Therefore $(X \vee Y) \wedge (X \vee Z) \subseteq X \vee (Y \wedge Z)$. Which means $X \vee (Y \wedge Z) = (X \vee Z) \vee (Y \wedge Z)$.
	
	The lattice $\mathfrak{L}$ is therefore a distributive orthocomplemented lattice, which means it is a Boolean algebra.
\end{proof}

	\begin{prop}
		Let $C \subseteq \Ens$ be a classical probability context. Let $\mathfrak{L}$ be the lattice of $\separate$-subspaces. Then the fraction capacity is an additive set function on the lattice. That is, for all $\ens \in C$ and $A, B \in \mathfrak{L}$ such that $A \cap B = \emptyset$,  $\fcap_{\ens}(A \vee B) = \fcap_{\ens}(A) + \fcap_{\ens}(B)$.
	\end{prop}
	
	\begin{proof}
		Let $A, B\in \mathfrak{L}$ such that $A \cap B = \emptyset$. Then $A$ is separate from $B$. Therefore $A \vee B$ is a classical probability context that is the convex closure of two separate probability contexts. Therefore, as we saw in a previous proof, $A \vee B$ consists of convex combinations of $A$, which are all in $A$, of convex combinations of $B$, which are all in $B$, and of convex combinations of both, which are in neither $A$ nor $B$.
		
		We want to show that $\fcap_{\ens}(A \vee B) = \fcap_{\ens}(A) + \fcap_{\ens}(B)$ if $\ens \in A \vee B$. Let $\ens \in A \vee B$ be an ensemble that is the mixture of elements of $A$. Then $\ens \in A$ and it has no components in $B$. Therefore $\fcap_{\ens}(A \vee B) = 1$ and $\fcap_{\ens}(A) = 1$ while $\fcap_{\ens}(B) = 0$. This means $\fcap_{\ens}(A \vee B) = 1 = 1 + 0 = \fcap_{\ens}(A) + \fcap_{\ens}(B)$. If $\ens$ is a mixture of elements of $B$, we get the same conclusion. The last case is when $\ens$ is a mixture of elements of $A$ and $B$. That is, $\ens = \sum_i p_i \ens[a]_i + \sum_j \lambda_j \ens[b]_j$ with $\ens[a]_i \in A$, $\ens[b]_j \in B$, $p_i, \lambda_j \in [0,1]$ and $\sum_i p_i + \sum_j \lambda_j = 1$. By definition of fraction capacity, $\sum_i p_i \leq \fcap_{\ens}(A)$. Suppose $\sum_i p_i < \fcap_{\ens}(A)$. Then there is $\hat{\ens[a]} \in A$ such that $\ens = \sum_i p_i \ens[a]_i + p \hat{\ens[a]} + \lambda \ens[c]$ for some $p,\lambda \in [0,1]$ and $\ens[c] \in A \vee B$. But then $\ens$ would be separately multidecomposable, which is a contradiction since it is an element of a classical probability context. Therefore $\sum_i p_i = \fcap_{\ens}(A)$. Similarly, we find that $\sum_j \lambda_j = \fcap_{\ens}(B)$. Since $\sum_i p_i + \sum_j \lambda_j = 1$, $\fcap_{\ens}(A) + \fcap_{\ens}(B) = 1 = \fcap_{\ens}(A \vee B)$ for all $\ens \in A \vee B$.
		
		Now let $\ens \in C$. We have $(A \vee B)^\separate \in \mathfrak{L}$ and $(A \vee B) \cap (A \vee B)^\separate = \emptyset$. Therefore $\fcap_{\ens}(C) = \fcap_{\ens}(A \vee B)+ \fcap_{\ens}((A \vee B)^\separate)$. Note that $\ens$, in general, is a convex combination of elements of $A\vee B$ and $(A \vee B)^\separate$. Components in $A \vee B$ are convex combinations of elements of $A$ and $B$, which also disjoint. Therefore $\ens$ is a convex combination of elements of $A$, $B$ and $(A \vee B)^\separate$, which are pairwise disjoint. As before, since multidecomposability is forbidden in a classical probability context, the sum of the coefficients for each part will have to match the fraction capacity of its subcontext. Therefore $\fcap_{\ens}(A) +  \fcap_{\ens}(B) + \fcap_{\ens}((A \vee B)^\separate) = \fcap_{\ens}(C) = \fcap_{\ens}(A \vee B) + \fcap_{\ens}((A \vee B)^\separate)$. Thus $\fcap_{\ens}(A \vee B) = \fcap_{\ens}(A) +  \fcap_{\ens}(B)$ for all $\ens \in C$.
	\end{proof}


\subsection{Probability measures from classical probability contexts}

The next step is to show that each ensemble in a classical probability context can be understood as a probability measure and vice-versa. The issue is how to recover the set of points, the spectrum $\sigma(C)$ of the context, and its topology and $\sigma$-algebra $\Sigma_{\sigma(C)}$. This is still an open problem. The Stone's representation theorem can be used as a fallback, though it does not recover the correct points.

\begin{defn}
	Let $C$ be a context. An \textbf{spectral element} of $C$ is a non-empty collection of subspaces $c \subset C$ such that
	\begin{enumerate}
		\item if $\{X_i\}_{i \in I} \in s$ then $\bigcap_{i \in I} X_i \in s$
		\item if $X \in s$ and $Y \in \mathfrak{L}$ such that $X \subseteq Y$, then $Y \in s$
		\item if $X \in s$ and $Y \in \mathfrak{L}$ such that $Y \subset X$ and $Y \neq \emptyset$, then there exists $Z \in s$ such that $Z \subset X$.
	\end{enumerate}
	The set of all elements of the context is called the \textbf{spectrum} of the context and is noted $\sigma(C)$.
\end{defn}

\begin{remark}
	The above should correspond to an \href{https://en.wikipedia.org/wiki/Ultrafilter}{ultrafilter}.
\end{remark}

\begin{defn}
	Let $C$ be a context and $X \in C$ be a subspace. A \textbf{spectral element} of $X$ is a spectral element $c \in \sigma(C)$ such that $X \in c$. The \textbf{spectral set} of $X$ is the collection of all its spectral elements. The standard topology of the spectrum $\sigma(C)$ is the one generated by the spectral sets of all subspaces $X \in C$.
\end{defn}

	\begin{defn}
		Given a poset $(P, \leq)$ downward refined filter is a proper filter $F$ such that if $x \in F$ and there exists a $y \in P$ such that $y < x$, then there exists a $z \in F$ such that $z < x$.
	\end{defn}
	
	\begin{defn}
		Given a classical probability context $C \subseteq \Ens$, the \textbf{spectrum} $\sigma(C)$ is the set of equivalence classes of sequences of ensembles that eventually become separate from each other. More rigorously, given a probability context, the spectrum is the collection of the downward refined filters of the lattice of $\separate$-subspaces $\mathfrak{L}^\separate(C)$. The spectrum of each $\separate$-subspace is given by $\sigma(A) \mapsto \{ x \in \sigma(C) \, | \, A \in x \}$. The standard topology of the spectrum is the one generated by the spectra of all $\separate$-subspaces (i.e. $\sigma(\mathfrak{L}^\separate(C))$.
	\end{defn}
	
	\begin{prop}
		Let $C \subseteq \Ens$ be a classical probability context and $\sigma(C)$ its spectrum. Then the following are true:
		\begin{enumerate}
			\item $A \subseteq B$ if and only if $\sigma(A) \subseteq \sigma(B)$
			\item $\exterior(\sigma(A)) = \sigma(A^{\separate})$
			\item $\partial \sigma(A) = \{ x \in \sigma(C) \, | \, A \notin x, A^{\separate} \notin x \}$
			\item $\sigma(A)^{\complement} = \overline{\sigma(A^{\separate})}$
			\item $\sigma(\bigwedge_{i \in I} A_i) = \interior\left(\bigcap_{i \in I} \sigma(A_i)\right)$
			\item $\sigma(\bigvee_{i \in I} A_i) = \interior\left(\overline{\bigcup_{i \in I} \sigma(A_i)}\right)$
			\item if $U$ open, then $U = \bigcup_{i \in I} \sigma(A_i)$ for some family of $A_i \in \mathfrak{L}^\separate(C)$
		\end{enumerate}
	\end{prop}
	
	\begin{proof}
		For 1, since all $x \in \sigma(C)$ are upward closed, $A \in x$ means $B \in x$ as well. Therefore if $x \in \sigma(A)$ then $x \in \sigma(B)$, which means $\sigma(A) \subseteq \sigma(B)$. Conversely, if $\sigma(A) \subseteq \sigma(B)$ then all downward sets of $\mathfrak{L}^\separate(C)$ that contain $A$ must also contain $B$, which means $B \supseteq A$.
		
		For 2 and 3, note that $\sigma(A) = \{ x \in \sigma(C) \, | \, A \in x \}$ and therefore $\sigma(A)^{\complement} = \{ x \in \sigma(C) \, | \, A \notin x \}$. Since $\sigma(A)$ is an open set, $\sigma(A)^{\complement} = \exterior(\sigma(A)) \cup \partial \sigma(A)$. Now consider $\sigma(A^{\separate})$. This is an open set and it is disjoint from $\sigma(A)$. In fact, if $x \in \sigma(A) \cap \sigma(A^{\separate})$ then $A, A^{\separate} \in x$. But this would mean that $\emptyset \in x$, which can't be because $x$ is a proper filter and cannot contain $\emptyset$. Since $A^{\separate}$ is the largest $\separate$-subspace that is separate from $A$, there is no set in $\sigma(\mathfrak{L}^\separate(C))$ that is larger than $A^{\separate}$ and still disjoint from $A$. Therefore $\sigma(A^{\separate}) = \exterior(\sigma(A))$. We have $\partial \sigma(A) = \sigma(C) \setminus (\interior(\sigma(A))\cup\exterior(\sigma(A))) = \{ x \in \sigma(C) \, | \, A \notin x, A^{\separate} \notin x \}$.
		
		For 4, since $\sigma(A)$ is an open set, the complement is the closure of the exterior. Therefore, given 2, $\sigma(A)^{\complement} = \overline{\sigma(A^{\separate})}$.
		
		For 5, consider $\bigwedge_{i \in I} A_i$. This is the largest $\separate$-subspace that is contained by all $A_i$. Then, given 1, $\sigma\left(\bigwedge_{i \in I} A_i\right)$ must be the largest open set that is contained by all $\sigma(A_i)$. This corresponds to $\interior\left(\bigcap_{i \in I} \sigma(A_i)\right)$.
		
		For 6, we have
		\begin{align*}
			\sigma(\bigvee_{i \in I} A_i) &= \exterior(\exterior(\sigma(\bigvee_{i \in I} A_i))) = \exterior(\sigma((\bigvee_{i \in I} A_i)^{\separate})) = \exterior(\sigma(\bigwedge_{i \in I} A_i^{\separate})) \\
			&= \exterior(\interior(\bigcap_{i \in I} \sigma(A_i^{\separate}))) = \exterior(\bigcap_{i \in I} \sigma(A_i^{\separate})) =  \\
			&= \exterior\left(\bigcap_{i \in I}\overline{\sigma(A_i)}^{\complement}\right) = \exterior\left(\left(\bigcup_{i \in I}\overline{\sigma(A_i)}\right)^{\complement}\right) \\
			&= \interior\left(\bigcup_{i \in I}\overline{\sigma(A_i)}\right)
		\end{align*}
		For the last step, note that the union of the closure is not necessarily the closure of the union, but they have the same interior.
		
		For 7, note that an open set $U$ is generated from $\sigma(\mathfrak{L}^\separate(C))$ through finite intersection and arbitrary union. Note that the finite intersection of open sets is an open set, so we have $\bigcap_{i \in I} \sigma(A_i) = \interior\left(\bigcap_{i \in I} \sigma(A_i)\right) = \sigma(\bigwedge_{i \in I} A_i)$. Therefore the finite intersection corresponds to a $\separate$-subspace. This means that we can can generate all open sets with arbitrary unions of sets from $\sigma(\mathfrak{L}^\separate(C))$.
	\end{proof}

\begin{coro}
	For every $A \in\sigma(\mathfrak{L}^\separate(C))$, $\sigma(A)$ is a regular open set. The topology of a spectrum is semiregular.
\end{coro}

\begin{proof}
	Since the lattice is orthocomplemented, we have $\sigma(A) = \sigma((A^{\separate})^{\separate}) = \exterior(\sigma(A^{\separate})) = \exterior(\exterior(A))$. Therefore $\sigma(A)$ is a regular open set.
	
	The topology is generated by regular open sets, and is therefore semiregular.
\end{proof}

\begin{conj}
	Given a point $x \in \sigma(C)$ and an open neighborhood $U$ of $x$, we can find a closed neighborhood $A$ that is a subset of $U$. The topology of a spectrum is regular.
\end{conj}

\begin{proof}
	Since $U$ is an open set, it is the union of a family of sets $\sigma(A_i)$. Since $U$ is a neighborhood of $x$, $x \in \sigma(A)$ with $A = A_i$ for some $i$. Now consider $y \in \partial \sigma(A)$. Since both $x$ and $y$ are two distinct downward refined filters, there will be a $\separate$-subspace $B_y$ such that $B_y \in x$ but $B_y \notin y$. That is, $x \in \sigma(B_y) \subseteq \sigma(A)$ and $y \notin \sigma(B_y)$. Now consider $D = \bigcap_{y \in \partial \sigma(A)} B_y$. Since $\mathfrak{L}^\separate(C)$ is an intersection structure, $D \in \mathfrak{L}^\separate(C)$. We also have $D \in x$ and therefore $D \neq \emptyset$. Therefore $\sigma(D)$ is an open set that is a subset of $\sigma(A)$ where we removed an open neighborhood around its boundary. This means that $\partial \sigma(D) \subseteq \sigma(A)$. This means that $\overline{\sigma(D)} \subseteq \sigma(A)$. Therefore we found a closed neighborhood of $x$ that is a subset of $U$.
	
	The above means that the closed neighborhoods of $x$ form a local base for $x$, which is another way to characterize regular topological spaces.
\end{proof}

	\begin{remark}
		Note how the infinite joins/meets on the lattice of the $\separate$-subspaces do not correspond to the infinite operations on the lattice of the topology or of the Borel sets. For example, suppose we are taking the lattice of probability measures defined over the interval $[0,1]$ that are absolutely continuous with respect to the Lebesgue measure. Each $\separate$-subspace will correspond to a set of measures whose support lives in a particular region. A singleton will not correspond to any subspace as it cannot support an absolutely continuous measure. Therefore, if we take all the $\separate$-subspaces of all measures that have $1/2$ within the support, the intersection of all those subspaces will be the empty set, which is a $\separate$-subspace. However, the intersection of the sets representing their support is the singleton $\{1/2\}$, which is not the empty set. However, its interior is the empty set.
		
		The fact that a single probability measure is an equivalence class of probability densities is related to this distinction. Consider a uniform distribution over $[0,1]$. The related probability density can be understood as a constant between those values and zero everywhere else. However, the same constant over $[0,1/2) \cup (1/2,1]$ will also work. That is, the same element of the ensemble space can be represented in different ways as a function over the spectrum.
	\end{remark}

\begin{conj}
	Let $C \subseteq \Ens$ be a probability context and $\ens \in C$ be an ensemble in the context. The set function $p_{\ens} : \Sigma_{\sigma(C)} \to[0,1]$, defined such that $p^*_{\ens}(U) = \inf(\{ \fcap_{\ens}(A) \, | \, \sigma(A) \supseteq U \})$, is a measure. Therefore each ensemble of a probability context is associated with a unique measure over the spectrum.
\end{conj}

\begin{conj}
	A convex subset $U \subset \Ens$ is a classical probability context if and only if it is the affine subspace of probability measures of a vector space of measures over a sample space $\Omega$.
\end{conj}

\begin{conj}
	Discrete and continuous classical ensemble spaces are classical probability contexts.
\end{conj}

A quantum ensemble space, however, does not allow a description in terms of classical probability precisely because it allows separate multidecompositions.

\begin{prop}
	A quantum ensemble space is not a classical probability context.
\end{prop}

\begin{proof}
	Let $\Ens$ be a quantum ensemble space. Let $A \subseteq \Ens$ the space of mixtures of a two dimensional subspace (i.e. a Bloch sphere) and consider the states $x^+$, $x^-$, $z^+$, and $z^-$, which are points on the surface that form a square. These are all separate ensembles. We have $\frac{1}{2} x^+ + \frac{1}{2} x^- = \frac{1}{2} z^+ + \frac{1}{2} z^-$ therefore the space allows separate multidecomposition and is not a classical probability context.
\end{proof}

However, given a maximal set of commuting observables, we can define a classical probability context as the set of all mixed states that commute with all the observables. This makes the spectrum of all the density operator coincide with the product of the spectra of all commuting observables, and therefore those mixed states can be characterized by a measure over the spectra.

\begin{conj}
	Let $\Ens$ be a quantum ensemble space. Let $O_i$ a maximal set of commuting observables. Let $C \subseteq \Ens$ be the set of mixed state that commute with all $O_i$. Then $C$ is a classical probability context.
\end{conj}


\section{Topological measures}

We still need to understand what requirements a measure must satisfy to describe a well-posed physical problem. The issue is that the set of all probability measures defined over a space is too broad, but it is not clear how it should be restricted.

If we consider the real line, given the Lebesgue measure $\mu$, \href{https://en.wikipedia.org/wiki/Lebesgue%27s_decomposition_theorem}{Lebesgue's decomposition theorem} assures us that every measure $\nu$ can be divided into these three components: an \href{https://en.wikipedia.org/wiki/Absolute_continuity#Absolute_continuity_of_measures}{absolutely continuous} part $\nu_{ac}$ with respect to the Lebesgue measure (i.e. $\nu_{ac}(U)=0$ for all sets $U$ for which $\mu(U)=0$), a pure point part $\nu_{pp}$ (i.e. there is a set of countable points $\{x_i\}$ such that $\nu_{pp}(\{x_i\}^{\complement})=0$) and a singular continuous part $\nu_{sc}$ (i.e. $\mu(\{x\})=0$ for all $x$ and there is a set $U$ such that $\mu(U)=0$ and $\nu_{sc}(U^{\complement})=0$).

To make this more concrete, let us assume that we are working on phase space, $\mu$ is the Liouville measure, which in statistical mechanics quantifies the states in a region, and $p$ is a probability measure. If $p$ is absolutely continuous with respect to $\mu$, it means that if a region has no states, it will have zero probability. This is the requirement under which the \href{https://en.wikipedia.org/wiki/Radon%E2%80%93Nikodym_theorem}{Radon-Nikodym theorem} applies and a probability density $\frac{dp}{d\mu}$ exists. Without this requirement, for example, the entropy cannot even be defined. On physics grounds, this requirement makes a lot of sense. A pure point measure would correspond to the case where the probability is concentrated into a few isolated points. In physics, these are often represented with delta functions. There is an inherent unphysicality of these measures: if we really have a continuum, it would make no sense to say that we are are able to prepare an exact value with certainty. We are essentially saying that we are able to concentrate the whole distribution on a set that is not experimentally verifiable (i.e. it is a closed set with no interior) and contains no states (i.e. the Liouville measure is zero). Yet, there are some cases where the distribution is over real values, but only certain values are allowed. For example, the mass spectrum for particles or the energy spectrum for a bound quantum system can only take certain values. In this case, the Lebesgue measure is the one that is meaningless, because the region in between the allowed value contains no physically meaningful cases. A singular continuous measure may be something physically irrelevant, like the \href{https://en.wikipedia.org/wiki/Cantor_distribution}{Cantor distribution} which is defined only on a Cantor set, but it may also represent a constrained distribution. For example, a uniform distribution over the surface of a sphere, if defined over three dimensional space, would have support on a measure zero set, and have zero probability at every point. The issue, again, is that the imposition of the constraint has to be applied to the whole mathematical structure, not just the probability measure.

While absolutely continuity is a requirement for a physically meaningful probability measure, it may not be enough. Consider a probability measure with a uniform distribution over a fat Cantor set. It would seem to be absolutely continuous but unlikely to be of physical interest.

Physically, the requirement of experimental verifiability poses constraints on what measures are possible, which means, mathematically, that there measures must satisfy some compatibility condition with the topology. In fact, it can be argued that some topological spaces may not allow any experimentally verifiable probability measure. Take, for example, the experimental domain generated by verifiable statements of the type \statement{there are at least $i$ elements.} That is, we can only verify the existence of some objects (e.g. fundamental particles) and not able to verify the non-existence of others. This gives us the natural numbers with the right topology. Can we, in line of principle, put a measure on this space? The problem is that, experimentally, we can only exclude a range below what was verified, so we cannot perform a repeated set of measurement and build a histogram. The only thing we would be able to confirm is the case that below a certain threshold we have probability zero.

It is not clear, however, what is the general rule. Naively, we thought that, given a verifiable statement, the non-termination condition (i.e. the boundary of the corresponding open set) cannot be assigned a non-zero measure. This intuition was reinforced by the finding that open sets have the feature that their boundary is nowhere dense (i.e. it has no interior), and sets with a nowhere dense boundary actually form a nice algebra. However, as we see in \ref{pm-es-nonZeroMeasureOnBoundaries}, there exists open sets that have boundaries with non-zero measures. Therefore, finding the right characterization is still an open problem.

We leave here the characterization of the algebra of sets with a nowhere dense boundary, in case it may be useful.


\subsection{Nicely boundaried sets}
Part of an original attempt to square measure and topology, sets with a nowhere dense boundary form a nice algebra that may, or may not, be useful.

\begin{mathSection}
	\begin{defn}
		Let $X$ be a topological space. A set $A \subseteq X$ is \textbf{nicely boundaried} if its boundary is nowhere dense.
	\end{defn}
	
	\begin{prop}
		The following are true:
		\begin{enumerate}
			\item open sets are nicely bounded
			\item the complement of a nicely bounded set is nicely bounded
			\item closed sets are nicely bounded
			\item the union of nicely bounded sets is nicely bounded
			\item the intersection of nicely bounded sets is nicely bounded.
		\end{enumerate}
		
		Open and closed sets are nicely bounded. The complement of a nicely bounded set is nicely bounded. The intersection and the union of nicely bounded sets are nicely bounded. The closure of open sets under complement, finite intersection and finite union is a Boolean algebra $R_X$ of nicely bounded sets.
	\end{prop}
	
	\begin{proof}
		For 1, let $A \subseteq X$ be an open set. We have $A = \interior(A)$ and, since $\partial A$ and $\interior(A)$ are disjoint, $\partial A$ and $A$ are disjoint. Let $U \subseteq \partial A$ be an open set. Then $U$ is disjoint from $A$ and therefore $U \subseteq \exterior(A)$. Since the boundary and the exterior are disjoint, $U$ must be the empty set.
		
		For 2, let $A$ be nicely bounded. Since $\partial A = \partial A^{\complement}$, $A^{\complement}$ is also nicely bounded.
		
		For 3, note that closed sets are complements of open sets, which are nicely bounded.
		
		For 4, let $A, B \subseteq X$ be two nicely bounded sets. The boundary \href{https://proofwiki.org/wiki/Boundary_of_Union_is_Subset_of_Union_of_Boundaries}{satisfies} the property $\partial (A \cup B) \subseteq \partial A \cup \partial B$. The union of two nowhere dense sets is nowhere dense and \href{https://proofwiki.org/wiki/Subset_of_Nowhere_Dense_Subset_is_Nowhere_Dense}{a subset of a nowhere dense set is nowhere dense}. Therefore $\partial (A \cup B)$ is nowhere dense and $A \cup B$ is a nicely bounded set.
		
		For 5, since complements and unions of nicely bounded sets are nicely bounded, by De Morgan intersections are nicely bounded as well.
	\end{proof}
	
	\begin{prop}
		Let $X$ be a topological space, the closure of open sets under complement, finite intersection and finite union is a Boolean algebra $R_X$ of nicely bounded sets.
	\end{prop}
	
	\begin{proof}
		By definition, $R_X$ is closed under complement, finite intersection and finite union and is therefore a Boolean algebra. Since open sets are nicely bounded, and the operation preserve the property, $R_X$ is composed of only nicely bounded sets.
	\end{proof}
	
	\begin{remark}
		Note that the algebra $R_X$ does not contain all the nicely bounded sets. Consider a circle in $\mathbb{R}^2$. Let $U$ be the open set of all the interior points and $B$ be the points on the circle at a rational angle. Now consider $A = U \cup B$: $U$ is the interior of $A$ and $B$ its boundary. The boundary is nowhere dense, therefore $A$ is a nicely bounded set. However, we cannot generate $A$ with finite operations from open sets of $\mathbb{R}^2$.
	\end{remark}
\end{mathSection}


\section{Spectrum of a quantity}

The definition of the spectrum of a quantity (i.e. the allowed values) is still an open problem. We leave here the latest attempt, more for inspiration and reference.

\begin{defn}
	Let $\Ens$ be an ensemble space and let $F : \Ens \to \mathbb{R}$ be a statistical quantity. Let $x,y \in \mathbb{R}$ such that $x \leq y$. We say $\ens[a]$ is \textbf{within $[x,y]$ of $F$} if $F(\ens) \in [x,y]$ for every component $\ens$ of $\ens[a]$. Analogous definitions apply for intervals bounded only on one side.
	
	We say $\ens[a] \in \Ens$ is \textbf{supported by $[x,y]$ of $F$} if:
	\begin{enumerate}
		\item $\ens[a]$ is within $[x,y]$ of $F$
		\item let $\ens$ be in the flat generated by $\ens[a]$ and all ensembles within $(-\infty,x]$, then $F(\ens) \leq F(\ens[a])$
		\item let $\ens$ be in the flat generated by $\ens[a]$ and all ensembles within $[y, +\infty)$, then $F(\ens[a]) \leq F(\ens)$.
	\end{enumerate}
	Analogous definition applies for intervals bounded only on one side.
	
	An \textbf{eigenstate} of $F$ is an ensemble $\ens[a] \in \Ens$ that is supported by a singleton $U=[a,a]$, where $a \in \mathbb{R}$ is called the respective \textbf{eigenvalue}. 
\end{defn}

\begin{prop}
	Let $\Ens$ be a discrete classical ensemble space over the set of points $X$, and $F$ a statistical quantity. Then $F(X)$ is the set of possible eigenvalues, and an eigenstate is any ensemble that can be expressed as a mixture of points with the same eigenvalue.
\end{prop}

\begin{proof}
	First we show that any extreme point $\ens[a] \in \Ens$ is an eigenstate with eigenvalue $F(\ens[a])$. First, we have $F(\ens[a]) \in [F(\ens[a]), F(\ens[a])]$, which satisfies the first condition for the support. Now let $\ens[b] \in \Ens$ be an ensemble within $(-\infty, F(\ens[a])]$. Let $\{\ens_i\}$ be the set of extreme points that are components of $\ens[b]$. Since $\{\ens_i\}$ are components of $\ens[b]$, $F(\ens_i) \leq F(\ens[a])$ since $\ens[b]$ is within $(-\infty, F(\ens[a])]$. Recall that orthogonality, for a classical space, coincides with disjoint support of the probability distribution. Also recall that, in a discrete classical ensemble space, all ensembles can be decomposed in terms of the extreme points. Then $\langle \{\ens[a], \ens[b]\} \rangle_{\ortho}$ is exactly $\hull(\{\ens_i\} \cup \{\ens[a]\})$. Given the linearity of $F$, for any convex combination $\ens$ of $\ens_i$ and $\ens[a]$ we have $F(\ens) \leq F(\ens[a])$. This satisfies the second condition for the support. The argument can be repeated with the upper bound, so the third condition is also satisfies. This means that any extreme point is $\ens[a]$ is an eigenstate of $F$ with eigenvalue $F(\ens[a])$.
	
	Note that the above argument works also if $\ens$ is a mixture of eigenstates with the same eigenvalues. 
	
	Lastly, we show that any ensemble that is a mixture of two extreme points with different eigenvalue is not an eigenstate.
	Let $\ens = p \ens[a] + \bar{p} \ens[b]$ be a mixture of two extreme points $\ens[a], \ens[b] \in \Ens$ such that $F(\ens[a]) \neq F(\ens[b])$. Without loss of generality, suppose $F(\ens[a]) < F(\ens[b])$. Given the linearity of $F$, we have $F(\ens[a]) < F(\ens) < F(\ens[b])$. Therefore $\ens$ is not within $[F(\ens),F(\ens)]$, is not supported by $[F(\ens),F(\ens)]$ and therefore is not an eigenstate. This also means that any ensemble of extreme points with different eigenvalue is not an eigenstate.
\end{proof}

\begin{prop}
	Let $\Ens$ be a continuous classical ensemble space over the symplectic manifold $X$, and $F$ a statistical quantity. Then $\rho \in \Ens$ is supported by $[x,y]$ of $F$ if and only if the $\int_{[x,y]} \rho d\mu = 1$. Moreover, $a \in \mathbb{R}$ is an eigenvalue if and only if there is an open set $U \subset X$ such that $f^{-1}(a) \supseteq U$ where $f : X \to \mathbb{R}$ is the state variable corresponding to $F$.
\end{prop}

\begin{proof}
	Let $F$ be a statistical quantity over a continuous classical ensemble space. Then we can find $f : X \to \mathbb{R}$ such that $F(\rho) = \int_X f \rho d\mu$ for all $\rho \in \Ens$. Given an interval $[x,y] \subseteq \mathbb{R}$, then we can define $U_{[x,y]} = f^{-1}([x,y])$ which can be understood as the set of particle states for which the value of the property is within the bounds. Any probability distribution $\rho$ can be decomposed as
	$$ \rho = p_{(-\infty, x]} \rho_{(-\infty, x]} + p_{[x,y]} \rho_{[x,y]} + p_{[y,-\infty)} \rho_{[y,-\infty)} $$
	where
	\begin{equation}
		\begin{aligned}
			p_{(-\infty, x]} &= \int_{U_{(-\infty, x]}} \rho d\mu &
			p_{[x,y]} &= \int_{U_{[x,y]}} \rho d\mu  &
			p_{[y,-\infty)} &= \int_{U_{[y,-\infty)}} \rho d\mu \\
			\rho_{(-\infty, x]} &= \frac{\left.\rho\right|_{U_{(-\infty, x]}}}{p_{(-\infty, x]}} &
			\rho_{[x,y]} &= \frac{\left.\rho\right|_{U_{[x,y]}}}{p_{[x,y]}} &
			\rho_{[y,-\infty)} &= \frac{\left.\rho\right|_{U_{[y,-\infty)}}}{p_{[y,-\infty)}}
		\end{aligned}
	\end{equation}
	
	Suppose the support of $\rho$ is within $U_{[x,y]}$. Then $\int_X \rho d\mu \in [x,y]$. Since any of its components will also have support within $U_{[x,y]}$, the expectation of $f$ for all its components is also within $[x,y]$, within means that $\rho$ is within $[x,y]$ of $F$. Conversely, if the support of $\rho$ is not within $U_{[x,y]}$, then we can find a component whose support is either within $U_{(-\infty, x]}$ or $U_{[y,-\infty)}$. Therefore, $\rho$ is within $[x,y]$ of $F$ if and only if the support of $\rho$ is within $U_{[x,y]}$.
	
	Now suppose $\rho$ is within 
	
	Now we show that $\rho$ is supported by $[x,y]$ of $F$ if $\rho_f(f) = 0$ for all $f \notin [x,y]$ where $\rho_f$ is the probability density function (PDF) of $F$. Since $\rho$ is a probability measure and $f$ is a random variable, we can define a PDF $\rho_f$. This can also be understood as the marginal of $\rho$ over $f$. If $\phi$ is a component of $\rho$, the corresponding $\phi_f$ will be a component of $\rho_f$. If $\rho$ is supported by $[x,y]$ of $F$, then the expectation of $f$ for any component of $\rho_f$ must be within $[x,y]$. This can only happen if the support of $\rho_f$ is within $[x,y]$, as any component below or above those bounds would have expectation below or above the bounds. Conversely, if the support of $\rho_f$ is within $[x,y]$ Therefore, $\rho$ is within $[x,y]$ of $F$ if $\rho_f(f) = 0$ for all $f \notin [x,y]$ where $\rho_f$
\end{proof}




\section{Expectation values and measure}

\subsection{Maximal component sequence}

\begin{defn}
	Let $\ens \in \Ens$ be an ensemble. A sequence of ensembles $\{\ens[a]_i\} \subseteq \Ens$ is an \textbf{increasing component sequence of $\ens$} if we can write
	\begin{align*}
		\ens &= p_i \ens[a]_i + \bar{p}_i \ens[b]_i  \\
		\ens[a]_{i+1} &= \frac{p_i}{p_{i+1}} \ens[a]_i + \frac{p_{i+1} - p_i}{p_{i+1}} \Delta \ens[a]_i
	\end{align*}
	where $\{\ens[b]_i\} \subseteq \Ens$, $\{\Delta\ens[a]_i\} \subseteq \Ens$ and $\{p_i\} \subseteq (0,1]$ is an increasing sequence. The \textbf{fraction} of the sequence is the limit $p_i \to p$.
\end{defn}

\begin{remark}
	Since the sequence of $p_i$ is increasing and is bounded, it must converge. The set of all possible limits is bounded and therefore must have a supremum.
\end{remark}

\begin{defn}
	Let $\ens \in \Ens$ be an ensemble and $A \subseteq \Ens$ a set of ensembles. Then the \textbf{$A$-components of $\ens$} are the components of $\ens$ that are mixtures of $A$. That is, $A_{\ens} = \{ \ens_1 \in \hull(A) \, | \, \exists p \in (0,1], \ens_2 \in \Ens \text{ s.t. } \ens = p \ens_1 + \bar{p} \ens_2  \}$. An \textbf{increasing $A$-component sequence of $\ens$} is an increasing component sequence of $\ens$ such that $\{\ens[a]_i\} \subseteq \hull(A)$ and $\{\Delta\ens[a]_i\} \subseteq \hull(A)$. The sequence $\{\ens[a]_i\} \subseteq \hull(A)$ is \textbf{maximal} if the fraction of the sequence is $\fcap_{\ens}(A)$.
\end{defn}

\begin{coro}
	The fraction of $A$-component sequences are bounded by $\fcap_{\ens}(A)$.
\end{coro}

\begin{proof}
	For all elements of component sequences we have $p_i \leq \fcap_{\ens}(A)$. Therefore the limit of $p_i$ cannot exceed $\fcap_{\ens}(A)$.
\end{proof}

\begin{prop}
	Let $\ens \in \Ens$ be an ensemble and $A \subseteq \Ens$ a set of ensembles, then there exists a maximal $A$-component sequence of $\ens$.
\end{prop}

\begin{proof}
	Consider the hull of $A$. This can be ordered by the fraction $\fraction_{\ens}$, which is less or equal to one. If there is a maximum, then simply take a sequence of an element with the maximum faction. If not, we can take a sequence of ever increasing fractions whose limit is the fraction capacity, and find a corresponding sequence of ensembles within $\hull(A)$. By definition, this will be a maximal $A$-component sequence of $\ens$.
\end{proof}

\begin{prop}
	Let $\{\ens[a]_i\} \subseteq \hull(A)$ be an $A$-component sequence of $\ens \in \Ens$. A \textbf{complement} sequence is an increasing sequence $\{\ens[b]_i\} \subseteq \Ens$ such that
	$$ \ens = p_i \ens[a]_i + q_i \ens[b]_i + \overline{(p_i + q_i)} \ens[\epsilon]_i $$
	where $\{\ens[\epsilon]_i\} \in \Ens$ and $(p_i + q_i) \to 1$. In this sense, the sum of the two sequences converges to $\ens$.
\end{prop}

\begin{remark}
	Note that complement sequences always exist since we can set $\ens[\epsilon]_i = \ens[b]_i$, $q_i = \bar{p}$ and recover the definition of a component sequence.
\end{remark}

\begin{prop}
	An $A$-component sequence $\{\ens[a]_i\} \subseteq \hull(A)$ of $\ens \in \Ens$ is maximal if and only if it admits a complement sequence that is always separate from the hull of A. That is, $\{\ens[b]_i\} \separate \hull(A)$.
\end{prop}

\begin{proof}
	Suppose that $\{\ens[a]_i\}$ is maximal and let $\{\ens[b]_i\}$ be a complement.
	$$ \ens = p_i \ens[a]_i + q_i \ens[b]_i + \overline{(p_i + q_i)} \ens[\epsilon]_i.$$
	
\end{proof}

\begin{prop}
	Let $\ens \in \Ens$ be an ensemble, $A \subseteq \Ens$ a set of ensembles and $\ens[a]_i$ and $\ens[b]_i$ two maximal $A$-component sequence of $\ens$. Then we can write $\ens[a]_i = p_i \ens[b]_i + \bar{p}_i \ens[c]_i$ where $\ens[c]_i \in \Ens$, $p_i \in [0,1]$ and $p_i \to 1$.
\end{prop}

\begin{proof}
	Note that we can always write $\ens[a]_i = p_i \ens[b]_i + \bar{p}_i \ens[c]_i$ because we can always choose $p_i = 0$ and $\ens[c]_i = \ens[a]_i$. Therefore, if the proposition is not true, we can always find $p_i \to p$, except that $p \neq 1$.
	
	Suppose the proposition is not true. We can still write $\ens = \lambda_i \ens[a]_i + \bar{\lambda}_i \ens[d]_i = \lambda_i p_i \ens[b]_i + \lambda_i \bar{p}_i \ens[c]_i + \bar{\lambda}_i \ens[d]_i$. But $\ens[b]_i$ is maximal, therefore $\lambda_i p_i \ens[b]_i$ can be increased. But $\ens[c]_i \in \hull(A)$, which means we can write an $A$-component sequence whose fraction is higher than the maximal, which cannot be. Therefore the proposition is true.
	
	TODO: this may need to be fixed. Can't reconstruct the argument.
\end{proof}

\begin{conj}
	Let $e \in \Ens$ be an ensemble, $F : \Ens \to \mathbb{R}$ be a statistical variable and $A \subset \Ens$ be a set of ensemble. Let $\ens[a]_i$ and $\ens[b]_i$ be maximal $A$-component sequences of $\ens$. Then $\lim_{i \to \infty} F(\ens[a]_i) = \lim_{i \to \infty} F(\ens[b]_i)$.
\end{conj}

\begin{defn}
	Let $e \in \Ens$ be an ensemble, $F : \Ens \to \mathbb{R}$ be a statistical variable and $A \subset \Ens$ be a set of ensemble. Then the \textbf{contribution to $F$ over $A$ given $\ens$} is the limit of variable for a maximal $A$-component sequence of $\ens$. That is, $F_{\ens}(A) = \lim_{i \to \infty} F(\ens[a]_i)$ where $\ens[a]_i$ is a maximal $A$-component sequence of $\ens$.
\end{defn}

\begin{prop}
	The contribution to $F$ is a set function.
\end{prop}

\begin{proof}
	Since $F_{\ens}$ takes a set of an argument and returns a real value, is a set function.
\end{proof}

\begin{remark}
	Note that the contribution cannot be monotone in the same way that the expectation of a random variable to an event is not monotone. Not clear whether the additivity of the variable may tell us something about the additivity of the contribution.
\end{remark}

\begin{conj}
	We can define a ``derivative'' $f_{\ens}$ between $F_{\ens}$ and $\fcap_{\ens}$ such that $F_{\ens}(A) = \int_A f_{\ens} \, d \fcap_{\ens}$ where $\int$ is the ??? integral.
\end{conj}

\begin{remark}
	This may still not be the correct formulation of the problem. Note that $A$ is a set of ensembles. In the classical case it would correspond to a set of probability measures, not a subset of the sample space (i.e. an event). However, in the discrete case and in the quantum case, $A$ can also be restricted to a set of pure state (i.e. extreme points), which would correspond to a subset of the sample space. It may be worth at least understanding that case.
\end{remark}


\section{Ensemble subspaces}

We now want to define a notion of subspace in a way that recovers the notions of subspaces we have in both classical and quantum mechanics. Given that orthogonality of subspaces in all three cases corresponds to ensembles saturate the upper entropy bound (i.e. orthogonal in the sense of the entropy), we will define the notion of subspaces based on the entropy.
\subsection{Ensemble subspaces}

We will now use the previous results where $X$ is an ensemble space and $R$ is the orthogonality relation between two ensembles.

\begin{prop}
	Let $\Ens$ be an ensemble space, orthogonality $\ortho$ is an irreflexive symmetric relation.
\end{prop}

\begin{proof}
	Since any ensemble mixed with itself saturates the lower bound of entropy, it does not satureate the upper bound and it is therefore not orthogonal with itself. Therefore orthogonality is irreflexive. Two elements are orthogonal if they satureate the upper entropy bound. This does not depend on their order. Therefore orthogonality is symmetric.
\end{proof}

\begin{defn}
	Let $\Ens$ be an ensemble space and $X \subseteq \Ens$ be a subset. The \textbf{orthogonal complement} $X^\ortho \subseteq \Ens$ is the set of all ensembles that are orthogonal from all elements of $X$. An \textbf{ensemble subspace} is a subset $X \in \Ens$ such that $X = (X^\ortho)^\ortho$.
\end{defn}

\begin{prop}\label{pm-es-subspacesContainHulls}
	Let $X \subseteq \Ens$ be an ensemble subspace. Then $\hull(U) \subseteq X$ for all $U \subseteq X$.
\end{prop}

\begin{proof}
	Let $U \subseteq X$. Then $U \ortho X^{\ortho}$. But since mixtures preserve orthogonality, we also have $\hull(U) \ortho X^{\ortho}$. Therefore $\hull(U) \subseteq X$.
\end{proof}

\begin{coro}
	Let $X \subseteq \Ens$ be an ensemble subspace, then $\hull(X) = X$.
\end{coro}

\begin{proof}
	Since $X \subseteq X$, $\hull(X) \subseteq X$ by \ref{pm-es-subspacesContainHulls}. By \ref{pm-es-hullProp}, we also have $X \subseteq \hull(X)$. Therefore $\hull(X) = X$.
\end{proof}

\begin{remark}
	The converse is not true: $\hull(X) = X$ does not mean $X$ is an ensemble subspace. For example, let $\Ens$ the ensemble space of a two state quantum system (i.e. the Bloch ball). Let $U$ be the set of all mixtures of two pure states (i.e. and the segment that connects two points on the surface). Then $U$ is closed under mixture (i.e. convex combinations) but it is not a subspace. In fact we have that $U^{\ortho}=\emptyset$ and therefore $(U^\ortho)^\ortho = \Ens$.
\end{remark}

\begin{prop}
	Let $\Ens$ be a discrete classical theory and $X \subseteq \Ens$ an ensemble subspace. Then $X$ is the set of all probability distributions over a subset $A$ of pure states.
\end{prop}

\begin{proof}
	Let $\Ens$ be a discrete classical ensemble space. Then each ensemble is a sequence $p_i$ such that $\sum_i p_i = 1$. The dimensionality of the space fixes the range of $i$. Note that, given that the space is discrete, the collection $p_i$ can be understood as a continuous function of the pure elements.
	
	Note that two probability distributions will be orthogonal if and only if their supports are disjoint. Therefore orthogonality for a discrete classical ensemble space is the same as having disjoint support. This means that a subspace is given by all possible probability distributions over a subset $A$ of all pure states.
\end{proof}

\begin{prop}
	Let $\Ens$ be a continuous classical theory and $X \subseteq \Ens$ an ensemble subspace. Then $X$ is a set of measures whose support is a regular closed set (i.e a set that is the closure of its own interior).
\end{prop}

\begin{proof}
	Let $\Ens$ be a continuous classical ensemble space. Then the probability density associated to each ensemble is a continuous function over a symplectic manifold $M$ that integrates to one. Since the function is continuous, it is non-zero on an open set, and the support is the closure of that set.
	
	Consider two continuous probability densities. They will be orthogonal if and only if they have disjoint support, meaning the interior of their supports are disjoint. Therefore orthogonality for a continuous classical ensemble space is the same as having disjoint support.
	
	Take a set of ensembles $U \subseteq \Ens$. An ensemble $\ens \in \Ens$ will be orthogonal from all ensembles in $U$ if and only if its support is disjoint from the union of all the supports of all the elements of $U$. Therefore $U^\ortho$ is the set of all ensembles whose support is within the closure of of the exterior of the union of all supports of elements of $U$. With a similar logic, $(U^\ortho)^\ortho$ is the set of all ensembles whose support is within the closure of the exterior of the union of the support of elements of $U^\ortho$. Therefore, $X$ will contain all probability measure whose support within a set $A \subseteq M$ that is the that $A = \overline{\interior(\interior(A))}$ and is therefore a regular set.
	
	Now take a regular closed set $A \subseteq M$ and let $X$ be the set of all continuous probability densities whose support is within $A$. Then $X^\ortho$ is the subspace of all continuous probability densities that have support within $\overline{\exterior(A)}$ and $(X^\ortho)^\ortho$ is the subspace of all continuous probability densities that have support within $\overline{\exterior(\exterior(A))} = \overline{\interior(A)} = A$. Therefore $X$ is a subspace.
\end{proof}

\begin{prop}
	Let $\Ens$ be a quantum ensemble space and $X \subseteq \Ens$ an ensemble subspace. Then $X$ is the set of mixed state whose support is a subspace of the corresponding Hilbert space.
\end{prop}

\begin{proof}
	Let $\Ens$ be a quantum ensemble space. Then each ensemble is a density operator defined over a Hilbert space $\mathcal{H}$. Consider two density operators. They will be orthogonal, that is the entropy increase is maximal, if and only if they are defined on orthogonal subspaces of $\mathcal{H}$. That is, $\ens_1 | \psi \> \neq 0$ only if $\ens_2 | \psi \> = 0$ and vice-versa. Therefore $X$ contains exactly all the mixed states whose support is a subspace of $\mathcal{H}$.
\end{proof}

\section{Alternative definition for contexts}

Alternatively, a context can be define in terms of a lattice of orthogonal sets.

\begin{defn}
	Let $\mathfrak{L}$ be the lattice of subspaces and $\wedge, \vee, (\cdot)^\ortho$ be respectively the join, meet and orthogonal complement. A \textbf{context} is a lattice of subspaces $C \subseteq \mathfrak{L}$ such that
	\begin{enumerate}
		\item if $\{X_i\}_{i \in I} \in C$ then $\bigwedge_{i \in I} X_i \in C$, $\bigvee_{i \in I} X_i \in C$ and $X_i^\ortho \in C$
		\item if $X, Y \in C$ and $X \cap Y = \emptyset$, then $X \ortho Y.$
	\end{enumerate}
	That is, the join, meet and complement operation in $\mathfrak{L}$ and $C$ are the same (i.e. smallest subspace that contains all, biggest subspace contained by all, orthogonal subspace). The additional property is that disjoint subspaces are orthogonal.
\end{defn}

\begin{prop}
	Let $C \subseteq \mathfrak{L}$ be a context. Then $\scap$ is an additive measure over the context. That is, $\scap(X \vee Y) = \scap(X) + \scap(Y)$ for all $X, Y, X \vee Y \in C$ such that $X \cap Y = \emptyset$.
\end{prop}

\begin{proof}
	Since $X \vee Y$ contains both $X$ and $Y$, we have $X \cup Y \subseteq(X \vee Y)$. By \ref{pm-es-subspacesContainHulls}, $\hull(X \cup Y) \subseteq(X \vee Y)$. By 
\end{proof}

\begin{defn}
	An ensemble $\ens \in \Ens$ is \textbf{compatible} with a context $C \subseteq \mathfrak{L}$ if it is orthogonally decomposable into an $X$-maximal sequence and $X^{\ortho}$-maximal sequence for any $X \in C$.
\end{defn}

\begin{coro}
	Let $\ens \in \Ens$ an ensembles compatible with a context $C \subset \mathfrak{L}$. Then $p_{\ens}(X^\ortho) = 1 - p_{\ens}(X)$.
\end{coro}

\begin{proof}
	Since $\ens$ is compatible with $C$, we can write $\ens = p_i \ens[x]_i + \lambda_i \ens[y] + \epsilon_i \ens_i$ where $\ens[x]_i \in X$ and $\ens[y]_i \in X^\ortho$ are maximal component sequences. Note that $p_{\ens}(X^\ortho) = 1 - p_{\ens}(X)$ if and only if $\epsilon_i \to 0$. Suppose it didn't. Then every $\ens_i$ would have a component that is neither in $X$ or $X^\ortho$. That would mean it is orthogonal from all the element of $X$ and of $X^\ortho$. But $X$ contains all the elements disjuct from $X^\ortho$ and vice-versa. Therefore such component cannot exist, $\epsilon_i \to 0$ and $p_{\ens}(X^\ortho) = 1 - p_{\ens}(X)$.
\end{proof}

\begin{conj}
	Let $C \subseteq \mathfrak{L}$ be a context and $\ens \in \Ens$ an ensemble compatible with the context. Then $p_{\ens}$ is an additive measure over the context. That is, $p_{\ens}(X \vee Y) = p_{\ens}(X) + p_{\ens}(Y)$ for all $X, Y, X \vee Y \in C$ and $X \cap Y = \emptyset$.
\end{conj}

\begin{remark}
	The additivity of probability is more of a special condition than one may expect. Naively, one would expect that if $U$ and $V$ are separate, then $p_{\ens}(U \cup V) = p_{\ens}(U) + p_{\ens}(V)$. That is, the component of $\ens$ in $U \cup V$ is simply the sum of the components within $U$ and $V$, which are going to be separate and orthogonal (i.e. will not overlap). However this is not true in quantum mechanics.
	
	Take a singe quibit and let $\ens$ be the maximally mixed state. Then $p_{\ens}(\{x\}) = \frac{1}{2}$ for any pure state $x$. However, if we take two pure states $x$ and $y$ that are not orthogonal, the maximally mixed state is not a mixture of $x$ and $y$, which means $p_{\ens}(\{x, y\}) < 1 = p_{\ens}(\{x\}) + p_{\ens}(\{y\})$.
\end{remark}

\begin{conj}
	An ensemble space $\Ens$ is a classical ensemble space if and only if $\mathfrak{L}$ is a context.
\end{conj}

\begin{conj}
	An ensemble space $\Ens$ is orthogonally decomposable if and only if every ensemble is compatible with some context.
\end{conj}

\fi