From fc1a4f4439bbea31e03d4dddbc45b4de4b0e91e1 Mon Sep 17 00:00:00 2001 From: heytitle Date: Thu, 26 Sep 2019 18:06:03 +0200 Subject: [PATCH 1/7] add a tutorial notebook --- python/notebooks/us-1994-census.ipynb | 1329 +++++++++++++++++++++++++ 1 file changed, 1329 insertions(+) create mode 100644 python/notebooks/us-1994-census.ipynb diff --git a/python/notebooks/us-1994-census.ipynb b/python/notebooks/us-1994-census.ipynb new file mode 100644 index 0000000..aa4cea7 --- /dev/null +++ b/python/notebooks/us-1994-census.ipynb @@ -0,0 +1,1329 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import pandas as pd\n", + "from matplotlib import pyplot as plt\n", + "import numpy as np\n", + "\n", + "import dit\n", + "import admUI\n", + "import math" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## General Structure (will be removed later)\n", + "\n", + "describe about the general structuure of this notebook. Ideally, it should contain\n", + "\n", + "- Problem definition\n", + "- Setting\n", + " - Data Preparation\n", + " - some statistics of us-census 1994\n", + " - auxiliary functions (helper) functions\n", + "\n", + "- Information theory\n", + " - basic quantities\n", + " - Entropy, KL-Divergence, Mutual Information, and some properites (chain rule, ... )\n", + " - information decomposision (BROJA & Pradeep's paper)\n", + " - MI = SI + CI + 2 UI\n", + " - breifly describe how the UI solver works. (simplex...), intuitively\n", + " - detail in appendix?" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Introduction\n", + "\n", + "Information and its theory are an important quantity that govern many fields, rangning from communication to machine learning. \n", + "\n", + "[figure src-> one receiver]\n", + "\n", + "In this tutorial, we are going to demonstate a way of decomposing information into parts. Each part describes a certain aspect of the relationship of the variables.\n", + "\n", + "[figure src-> two receiver]\n", + "\n", + "We start the turotial with dataset description and some preprocessing steps ([Section X](#Data-Preparation)). Then, we define some auxiliary functions for visualizing results. Then, we talk about the foundation of information theory (Section X) and Information Decomposition. Along the way, we have code that implement or compute quantities of current interest." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Data Preparation\n", + "\n", + "In this tutorial, we use **[US Census 1994][us-census]**, a publily available dataset, to demonstate the content of this turorial. The dataset contains individual's attributes, such as race, age, gender, and the level of income ( <= X, > X ). Hence, ML learners use it to train a classifier for predicting the level of income based on other attributes.\n", + "\n", + "\n", + "Here, we are interested in explaing the relationship between these attributes and the income variable; therefore, we only use the training set.\n", + "\n", + "[us-census]: https://archive.ics.uci.edu/ml/datasets/adult" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "data_url = \"https://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.data\"" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "column_names = [\n", + " \"age\",\n", + " \"workclass\",\n", + " \"fnlwgt\",\n", + " \"education\",\n", + " \"education-num\",\n", + " \"martital-status\",\n", + " \"occupation\",\n", + " \"relationship\",\n", + " \"race\",\n", + " \"sex\",\n", + " \"capital-gain\",\n", + " \"capital-loss\",\n", + " \"hours-per-week\",\n", + " \"native-country\",\n", + " \"income\",\n", + "]\n", + "\n", + "\n", + "df = pd.read_csv(data_url, names=column_names)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Cleansing" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
ageworkclassfnlwgteducationeducation-nummartital-statusoccupationrelationshipracesexcapital-gaincapital-losshours-per-weeknative-countryincome
039State-gov77516Bachelors13Never-marriedAdm-clericalNot-in-familyWhiteMale2174040United-States<=50K
150Self-emp-not-inc83311Bachelors13Married-civ-spouseExec-managerialHusbandWhiteMale0013United-States<=50K
238Private215646HS-grad9DivorcedHandlers-cleanersNot-in-familyWhiteMale0040United-States<=50K
353Private23472111th7Married-civ-spouseHandlers-cleanersHusbandBlackMale0040United-States<=50K
428Private338409Bachelors13Married-civ-spouseProf-specialtyWifeBlackFemale0040Cuba<=50K
\n", + "
" + ], + "text/plain": [ + " age workclass fnlwgt education education-num \\\n", + "0 39 State-gov 77516 Bachelors 13 \n", + "1 50 Self-emp-not-inc 83311 Bachelors 13 \n", + "2 38 Private 215646 HS-grad 9 \n", + "3 53 Private 234721 11th 7 \n", + "4 28 Private 338409 Bachelors 13 \n", + "\n", + " martital-status occupation relationship race sex \\\n", + "0 Never-married Adm-clerical Not-in-family White Male \n", + "1 Married-civ-spouse Exec-managerial Husband White Male \n", + "2 Divorced Handlers-cleaners Not-in-family White Male \n", + "3 Married-civ-spouse Handlers-cleaners Husband Black Male \n", + "4 Married-civ-spouse Prof-specialty Wife Black Female \n", + "\n", + " capital-gain capital-loss hours-per-week native-country income \n", + "0 2174 0 40 United-States <=50K \n", + "1 0 0 13 United-States <=50K \n", + "2 0 0 40 United-States <=50K \n", + "3 0 0 40 United-States <=50K \n", + "4 0 0 40 Cuba <=50K " + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# data exploration\n", + "\n", + "df[:5]" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[39, ' State-gov', 77516, ' Bachelors', 13, ' Never-married',\n", + " ' Adm-clerical', ' Not-in-family', ' White', ' Male', 2174, 0,\n", + " 40, ' United-States', ' <=50K']], dtype=object)" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df[:1].values" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As you can see from the above, some attributes (columns) contains a space in the begining. Although these spaces do not\n", + "affect our computation, it is still good to clean it up. \n", + "\n", + "This can been done by finding string columns (stored as `object`) and use Python's `strip` function to remove these prefix spaces." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "for k, v in zip(column_names, df.dtypes):\n", + " if v == \"object\":\n", + " df[k] = df[k].apply(lambda x: x.strip())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Discretization\n", + "\n", + "As you can see from the data exploration part, `age` and `hours-per-week` are continuous. This might be useful for some cases to treat them as they are. In this tutorial, we are interested in only groups of these values. Therefore, we need first need to perform discretization on this values. More precisely,\n", + "\n", + "- We categorize `age` into four groups: [ '<24', '24-35', '36-50', '>50' ], and\n", + "- We group `hours_per_week_group` into two groups: ['<=40', '>40']" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "age_groups = ['<24', '24-35', '36-50', '>50']\n", + "df['age-group'] = df.age.apply(lambda x: age_groups[np.digitize(x, [24, 36, 51])])\n", + "\n", + "hours_per_week_group = ['<=40', '>40']\n", + "df['hours-per-week-group'] = df['hours-per-week'].apply(lambda x: hours_per_week_group[0 if x <= 40 else 1])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Data Distribution\n", + "\n", + "Once we have finished cleansing and discretizing the data, we are now ready to instantiate a [dit][dit]'s `Distribution` variable. This variable comes with necessary methods for dealing probabilistic operators, such as marginalization and conditioning.\n", + "\n", + "[dit]: https://github.com/dit/dit" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "# select only columns that we will use in this tutorial\n", + "selected_columns = [\n", + " 'income',\n", + " 'education',\n", + " 'sex',\n", + " 'race',\n", + " 'occupation',\n", + " 'age-group',\n", + " 'hours-per-week-group'\n", + "]\n", + "\n", + "# aliases\n", + "rvs_names = [\n", + " 'S', # income\n", + " 'E', # education\n", + " 'G', # sex\n", + " 'R', # race\n", + " 'O', # occupation\n", + " 'A', # age\n", + " 'H', # hours-per-week\n", + "]\n", + "\n", + "rvs_to_name = dict(zip(rvs_names, selected_columns))" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "# take all samples with attributes that we're interested in \n", + "data_array = list(map(lambda r: tuple(r[k] for k in selected_columns), df.to_dict(\"record\")))\n", + "\n", + "# create distribution from the samples with uniform distribution\n", + "dist_census = dit.Distribution(data_array, [1. / df.shape[0] ] * df.shape[0])\n", + "\n", + "# set variable aliases to the discribution\n", + "dist_census.set_rv_names(\"\".join(rvs_names))" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
Class:Distribution
Alphabet:('Female', 'Male') for all rvs
Base:linear
Outcome Class:tuple
Outcome Lenght:1
Gp(x)
Female0.33079450876815825
Male0.6692054912318418
" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# marginal distribution of P(G), G is `sex`\n", + "dist_census.marginal('G')" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "-----------------------\n", + "Marginal: ('Female',) | 0.330795\n", + "P(S|H='Female')\n", + "Class: Distribution\n", + "Alphabet: ('<=50K', '>50K') for all rvs\n", + "Base: linear\n", + "Outcome Class: tuple\n", + "Outcome Length: 1\n", + "RV Names: ('S',)\n", + "\n", + "x p(x)\n", + "('<=50K',) 0.8905394113824158\n", + "('>50K',) 0.10946058861758425\n", + "\n", + "-----------------------\n", + "Marginal: ('Male',) | 0.669205\n", + "P(S|H='Male')\n", + "Class: Distribution\n", + "Alphabet: ('<=50K', '>50K') for all rvs\n", + "Base: linear\n", + "Outcome Class: tuple\n", + "Outcome Length: 1\n", + "RV Names: ('S',)\n", + "\n", + "x p(x)\n", + "('<=50K',) 0.6942634235888022\n", + "('>50K',) 0.3057365764111978\n" + ] + } + ], + "source": [ + "# conditional probablity P(S|G).\n", + "marginal, cdists = dist_census.condition_on('G', rvs='S')\n", + "\n", + "for i, (c, d) in enumerate(zip(cdists, marginal.zipped())):\n", + " print(\"\")\n", + " print(\"-----------------------\")\n", + " print(\"Marginal: %s | %f\" % d)\n", + " print(\"P(S|H='%s')\" % (d[0]))\n", + " print(c)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Basic Information Theory\n", + "\n", + "[note: revise this paragraph, make it relevant to what we're trying to do]\n", + "\n", + "Information theory is foundation of many fields and technologies. The theory provides rigourous methods that enable us to develop ways of communication between source and receivers via **noisy** channels with least amount of error. \n", + "\n", + "Some important quantities in Information Theory are:\n", + "\n", + "## Entropy\n", + "Let $X$ be a discreate random variable. Random variable $X$ takes values from $\\mathcal{X}$ and has probability mass function $p(x) = P\\{X=x\\}, x \\in \\mathcal{X} $.\n", + "\n", + "**Definition:** Entropy $H(X)$ of a discrete random variable $X$ is defined by\n", + "\n", + "$$\n", + "H(X) = - \\sum_{x \\in \\mathcal{X} } p(x) \\log_{2} p(x).\n", + "$$\n", + "\n", + "Because of $\\log_2$ in the equation, it is measured in terms of *bits*, and we omit writing the base of the log from now onwards." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "0.9157360598501509" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# H(G)\n", + "p_G = dist_census.marginal('G')\n", + "dit.shannon.entropy(p_G)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Joint Entropy and Contional Entropy\n", + "\n", + "Let $(X, Y)$ be a pair of discrete random variable with a joint distribution $p(x,y)$, $x \\in \\mathcal{X}$ and $y \\in \\mathcal{Y}$.\n", + "\n", + "**Definition:** Joint Entropy $H(X,Y)$ is definied by\n", + "\n", + "$$\n", + "H(X, Y) = - \\sum_{x \\in \\mathcal{X}} \\sum_{y \\in \\mathcal{Y}} p(x, y) \\log p(x, y)\n", + "$$" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "1.674948627614043" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# H(S, G)\n", + "p_SG = dist_census.marginal('SG')\n", + "dit.shannon.entropy(p_SG)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Definition:** Conditional Entropy $H(Y|X)$ is defined by \n", + "\n", + "$$\n", + "H(Y|X) = - \\sum_{x \\in \\mathcal{X}} \\sum_{y \\in \\mathcal{Y}} p(x, y) \\log p(y|x)\n", + "$$" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "0.7592125677638922" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# H(S|G)\n", + "dit.shannon.conditional_entropy(dist_census, 'S', 'G') " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Theorem:** Chain Rule of Joint Entropy $H(X, Y)$\n", + "\n", + "$$\n", + "H(X, Y) = H(X) + H(Y|X)\n", + "$$\n", + "\n", + "Proof can be found at [Cover, Thomas M and Thomas, Joy A's Elements of Information Theory, Theorem 2.2.1][element].\n", + "\n", + "Below, we verify the theorem computationally.\n", + "\n", + "[element]: https://www.wiley.com/en-it/Elements+of+Information+Theory,+2nd+Edition-p-9780471241959" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "dit.shannon.entropy(p_SG) \\\n", + " == dit.shannon.entropy(p_G) + dit.shannon.conditional_entropy(dist_census, 'S', 'G') " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Relative Entropy\n", + "\n", + "Let $p$ and $q$ be two distributions with probability mass function $p(x)$ and $q(x)$ accordingly. Relative entropy is a distance between $p$ and $q$.\n", + "\n", + "**Definition:** Relative entropy between two distributions $p$ and $q$ $D(p\\|q)$ is defined by \n", + "\n", + "$$\n", + "D(p\\|q) = \\sum_{x \\in \\mathcal{X}} p(x) \\log \\frac{p(x)}{q(x)}\n", + "$$\n", + "\n", + "Another name of relative entropy is **Kullback-Leibler** divergence. Important properties of $D(p\\|q)$ are:\n", + "- $D(p\\|q) \\neq D(q\\|p)$\n", + "- $D(p\\|q) = 0$ iff. $p=q$ \n", + "- $D(p\\|q) = \\infty $ if there is some $x \\in \\mathcal{X}$ that $p(x) > 0$ and $q(x) = 0$.\n", + "\n", + "## Mutual Information\n", + "\n", + "For two discrete random variables $X$ and $Y$ with a joint probability $p(x, y)$ and marginal probablity mass function $p(x)$ and $p(y)$. \n", + "\n", + "\n", + "**Definition:** Mutual information $I(X; Y)$ is the relative entropy between $p(x, y)$ and $p(x)p(y)$: \n", + "\n", + "$$\n", + "\\begin{align*}\n", + "I(X; Y) &= D \\big(p(x, y) \\| p(x)p(y) \\big) \\\\\n", + "&= \\sum_{x \\mathcal{X}}\\sum_{y \\in \\mathcal{Y}} p(x, y) \\log \\frac{p(x, y)}{p(x)p(y)}\n", + "\\end{align*}\n", + "$$\n", + "\n", + "[note: Remove this part below]\n", + "If we continute the derivation, we have \n", + "\n", + "$$\n", + "I(X; Y) = H(X) - H(X|Y)\n", + "$$\n", + "\n", + "Intuitively, $I(X; Y)$ describes the overvall uncertainty of $X$ when $Y$ is known to us or vice versa. Please see [note: Appendix X][continue] for the derivation.\n", + "\n", + "[continue]: ..." + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "0.037171387438320824" + ] + }, + "execution_count": 36, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "dit.shannon.mutual_information(dist_census, 'S', 'G')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Contitional Mutual Information\n", + "\n", + "**Definition:** For three discrete random variables $X, Y, Z$, the conditional mutual information $I(X; Y |Z)$ is defined by\n", + "\n", + "$$\n", + "I(X; Y |Z) = H(X|Z) - H(X|Y,Z).\n", + "$$" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "0.02187436337675841" + ] + }, + "execution_count": 17, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# I(S, H | O)\n", + "dit.shannon.conditional_entropy(dist_census, 'S', 'O') - \\\n", + " dit.shannon.conditional_entropy(dist_census, 'S', 'HO')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Information Decomposition\n", + "\n", + "Let consider the setting that we have three random variables $X, Y, Z$. We are interested in knowing about $X$, but it is not observable. We can only observe the values of $Y$ and $Z$. \n", + "![](https://i.imgur.com/GpHQ6MW.png)\n", + "\n", + "Utimately, we would like to quantify how much we know about $X$ based on the information of $Y$ and $Z$. More precisely, $I(X; (Y, Z))$ is the total information of $X$ that $(Y, Z)$ contains.\n", + "\n", + "\n", + "[Bertschinger et al. (2014)][paper] proposes one approach to decompose $I(X; (Y, Z))$ into four quantities:\n", + "\n", + "\n", + "![](https://i.imgur.com/c3tEced.png)\n", + "\n", + "$$\n", + "I(X; Y, Z) = SI(X; Y, Z) + CI(X; Y, Z) + UI(X; Y \\backslash Z) + UI(X; Z \\backslash Y),\n", + "$$\n", + "\n", + "where \n", + "- $SI(X; Y, Z)$ is shared information that both $Y$ and $Z$ have about $X$,\n", + "- $CI(X; Y, Z)$ is complimentary (synergic) information that $Y$ and $Z$ have about $X$ when considering them together,\n", + "- $UI(X; Y \\backslash Z)$ is unique information that only $Y$ has about $X$ (in respect to $Z$), and vice versa. \n", + "\n", + "With the formulation above, these four quantities have to be non-negative.\n", + "\n", + "## Shared Information\n", + "Furthermore, we have the following equilities:\n", + "\n", + "$$\n", + "\\begin{align*}\n", + " SI(X; Y, Z) = I(X; Y) - UI(X; Y \\backslash Z) = I(X; Z) - UI(X; Z \\backslash Y) \\\\\n", + "\\end{align*}\n", + "$$\n", + "\n", + "\n", + "[paper]: https://www.mdpi.com/1099-4300/16/4/2161" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Co-Information\n", + "\n", + "**Definition:** Previouly known as **interaction information** [note: cite], co-Information $CoI(X;Y;Z)$ is defined by\n", + "\n", + "$$\n", + "CoI(X;Y;Z) = I(X;Y) - I(X;Y|Z).\n", + "$$\n", + "\n", + "With the chain rule of mutual information, we can write the identity above as\n", + "\n", + "$$\n", + "CoI(X;Y;Z) = SI(X; Y, Z) - CI(X; Y, Z).\n", + "$$\n" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "-0.00010688124181967851" + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# compute CoI(S; E; G) using dit\n", + "dit.multivariate.coinformation(dist_census, 'SEG')" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "0.012185581661140699" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# compute CoI(S; H; G) using dit\n", + "dit.multivariate.coinformation(dist_census, 'SHG')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Unique Information\n", + "\n", + "From above, we have everything in place except $UI(\\cdot)$ that we haven't defined yet. Bertschinger et al. (2014) define the unique information as follows:\n", + "\n", + "$$\n", + "UI(X; Y \\backslash Z) = \\min_{Q \\in \\Delta_p} I_Q(X; Y|Z), \n", + "$$\n", + "\n", + "where $\\Delta_p$ is a set of joint probability distributions that have the same marginal probablity distributions on $(X, Y)$ and $(X, Z)$ as $P \\in \\mathbb{P}_{ \\mathcal{X} \\times \\mathcal{Y} \\times \\mathcal{Z} }$.\n", + "\n", + "Formally, $\\Delta_p$ is\n", + "\n", + "$$\n", + "\\Delta_p = \\{ Q \\in \\mathbb{P}_{ \\mathcal{X} \\times \\mathcal{Y} \\times \\mathcal{Z} }: Q(X, Y) = P(X, Y) \\text{ and } Q(X, Z) = P(X, Z) \\}\n", + "$$\n", + "\n", + "[todo: add figure to illustrate this optimization procedure, maybe in appendix]" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [], + "source": [ + "# find q for S, G, R (income, race, gender)\n", + "q_SGR = admUI.computeQUI(distSXY = dist_census.marginal('SGR'))\n", + "\n", + "# due to the fact that computeQUI rename variables to SXY\n", + "# we need to rename them back to SRG\n", + "q_SGR.set_rv_names(\"SGR\")" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
Class:Distribution
Alphabet:(('<=50K', '>50K'), ('Female', 'Male'), ('Amer-Indian-Eskimo', 'Asian-Pac-Islander', 'Black', 'Other', 'White'))
Base:linear
Outcome Class:tuple
Outcome Lenght:3
SGRp(x)
<=50KFemaleAmer-Indian-Eskimo0.008247547005324936
<=50KFemaleAsian-Pac-Islander0.005815377888949708
<=50KFemaleBlack0.07930424583381772
<=50KFemaleOther0.007555050560623192
<=50KFemaleWhite0.1936633251347222
<=50KMaleAmer-Indian-Eskimo0.0001981395952810085
<=50KMaleAsian-Pac-Islander0.01761756333138691
<=50KMaleBlack0.00475336950884044
<=50KMaleWhite0.44203582369502953
>50KFemaleAmer-Indian-Eskimo0.001018361885656181
>50KFemaleAsian-Pac-Islander0.0007180555469803716
>50KFemaleBlack0.009792116811899728
>50KFemaleOther0.0007677896847523185
>50KFemaleWhite0.023912637763622506
>50KMaleAmer-Indian-Eskimo8.725526069439487e-05
>50KMaleAsian-Pac-Islander0.007758342599998848
>50KMaleBlack0.0020932675154360938
>50KMaleWhite0.19466173037698395
" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "q_SGR" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "0.028813064495489815" + ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# H(S|G,R)\n", + "h_SgGR = dit.shannon.conditional_entropy(q_SGR, 'S', 'GR')\n", + "\n", + "# UI(S; G \\ R) = I_Q { S; G \\ R }\n", + "ui_SG_R = dit.shannon.conditional_entropy(q_SGR, 'S', 'R') - h_SgGR\n", + "\n", + "ui_SG_R" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "1.9623869407237038e-05" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# UI(S; R \\ G) = I_Q { S; R \\ G }\n", + "ui_SR_G = dit.shannon.conditional_entropy(q_SGR, 'S', 'G') - h_SgGR\n", + "\n", + "ui_SR_G " + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# compute shared information\n", + "si_SGR = dit.shannon.mutual_information(q_SGR, 'S', 'G') - ui_SG_R \n", + "si_SRG = dit.shannon.mutual_information(q_SGR, 'S', 'R') - ui_SR_G\n", + "\n", + "# sanity check: by the definition of shared information\n", + "si_SGR == si_SRG" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "0.0" + ] + }, + "execution_count": 25, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# by the definition of co-information\n", + "ci_SGR = si_SRG - dit.multivariate.coinformation(q_SGR, 'SGR')\n", + "ci_SGR" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 26, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# sanity check: by the definition of Bertschinger et al. (2014)'s information decomposition\n", + "dit.shannon.mutual_information(q_SGR, 'S', 'GR') == ui_SG_R + ui_SR_G + si_SRG + ci_SGR " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Result Visualization" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [], + "source": [ + "# define measurement metrics\n", + "metric_keys = ['si', 'ci', 'ui_0', 'ui_1']\n", + "\n", + "# define legend labels\n", + "latex_labels = [\n", + " '$SI$',\n", + " '$CI$',\n", + " '$UI(S; Y \\\\backslash Z)$',\n", + " '$UI(S; Z \\\\backslash Y)$'\n", + "]\n", + "\n", + "# define colour for ploting\n", + "colors = [\n", + " '#27AB93',\n", + " '#FF5716',\n", + " '#D33139',\n", + " '#522F60',\n", + "] " + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgwAAABXCAYAAABoUB0SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAXFUlEQVR4nO3de1SU550H8O8PBmW4qgFv8YIFgRAQRCBl6wnxlAASBTXERmzTeFJ3jWatG+Oe5KSeY7Pt7rYx2RiaC+omtu4i9bIxWk2D2zS0JgQC4sCESBwjKCgGI3IJTGFmnv2DwR1wYMZwGcHv5xyOM8/7zDu/3wwyv3me930fUUqBiIiIaCBurg6AiIiIbn8sGIiIiMghFgxERETkEAsGIiIicogFAxERETnEgoGIiIgc0rg6ACIiosEoKyubrNFodgOIBL8If1sWAHqTyfSTBQsWfGWvAwsGIiIa1TQaze6pU6feExgY2OTm5saLC30LFotFGhsbIxoaGnYDyLDXh5UYERGNdpGBgYEtLBa+PTc3NxUYGNiM7lEa+31GMB4iIqLh4MZiYfCsr2G/dQELBiIiInKIBQMRERE5xIKBiIhoCNTW1nosWbLkO5GRkfcEBQVF3nfffaEA8OKLLwZkZ2fPdnV8g8WzJIiIaEwJP/ibBUO5vzNZT5U502/VqlVz1qxZ07h27domACgpKdECQGVlpVdUVFT7UMbkChxhICIiGiSTyYSSkhLf1NTU1p62hISEDgCoqqrSxsTEdLguuqHBgoGIiGiQNBoNEhMTW+bPn39vdnb27IKCAu+ebWfPntXGx8ezYCAiIiKgsLDwbF5e3jk/Pz/z8uXLQ/fu3TvBYDB4aLVaS0BAgNnV8Q0Wj2EgIiIaAm5ubkhNTW1LTU1tu379urtOp9OOGzdOhYeHj/rjFwCOMBAREQ3aoUOH/IxGowBAfX29pqSkxCctLa1Fp9NpIyIiRv10BMCCgYiIaNAOHDgwce7cufeGhYVFpKWlzd26deul5OTkb/R6vXbevHljomAQpXg1TSIiGr10Ol1NdHT0VVfHMRbodLqA6OjoIHvbOMJAREREDrFgICIiIodYMBAREZFDLBiIiIjIIRYMRERE5BALBiIiInKIBQMRERE5xIKBiIiIHOJaEi4SEBCggoKCXB0GEdGoUlZWdlUpFejqOPrz1ltvTXzllVemdHZ2uplMJsnMzLw2derUrvLycu+8vLxaV8c3GCwYXKTN3wttzz7u6jBuWwf/Nc/VIRDRMIg89fGgHi8ijj90H52yYFBP0lf+lTJnuuXk5NyVm5s7+d133zUEBwd3tba2ur366qsBlZWVXlFRUaN+ASpOSRAREQ3StWvX3LZu3Tpz//7954KDg7sAwNfX1/L8889/VVVVpY2JiRn160mwYCAiIhqkvLy8idHR0W0RERGdfbedPXtWGx8fz4KBiIjoTqfX67VRUVE3FQUGg8FDq9VaAgICzK6IayixYCAiIhokb29vs8Viuam9rKzMKzw8fNQfvwCwYCAiIhq0JUuWNB89enTixYsXNQDQ0dEhL730UoBOp9NGRESM+ukIgGdJEBERDdqiRYvan3322UsPPvhgqMVigclkkkceeeTrzz//XJuent7s6viGAgsGIiIaW5w8DXKobdiw4dqGDRuuueK5RwKnJIiIiMghlxQMIvKKiNwvIr8UkV/ZtM8WkS9FZIKI5IvIXFfEZ49NzDNF5HSfn5aePETkf0VkoqvjJSIiGkojXjCIyF0AvquU+guAXwBYJiL3WDfvALBVKXUdwBsA/vkW9z0sUyy2MSulLiqlYnp+APwIQDOA/7B23wtg/XDEQURE5CoDFgwi4i0ix0REJyJ6EfmBtX2BiBSKSJmIvC8i00REIyKfisgD1j7/JiK/tLPbhwH8EQCUUh0A/gnAayKSDsBXKfXf1n5/BZDsqAgQkQ+t3/5LAfxURJaKSLGIlFu/7U+x9vMRkbdFpFJEKkTkYWt7iogUicgpETkgIj4DxdznuT0B5AHYoJRqsDYfAbBqoJiJiIhGG0cjDGkALimlopVSkQD+KCIeAHIAZCmlFgB4C8AvlVImAI8DeENEkq2P/bmdfX4PwI0DUpRSxwE0AfgtbL6ZK6UsAAwAop3IY5xSKk4p9RKAk+geDZgPIB//P0qxFUCzUipKKTUPwAciEgDgZwCSlVKxAEoBPO0oZhu/BnBSKXXEJu4mAOOtoxK9iMjfi0ipiJSaW9qcSIuIiOj24GgIvxLAS9b5+T8opf4qIpEAIgGcEBEAcAdwGQCUUp+JyF4AfwCQqJS66RKZAKYBaOzT9hoArVKquk/7VwCmw/6Hta3f29yeAeD3IjINwDgA563tyQAe7emklGoSkSUAIgB8ZM1lHIAiZ2IWkcXWfdpb5KQn7q9tG5VSOwHsBADP4FnKQU5ERES3jQELBqXUFyISCyAdwC9E5E8A3gHwmVIqsZ+HRQG4DmByP9s7AHj2abNYf/rytPZ35Bub2zkAXlZKHbFOj2wb4HEC4IRSytEUQq+YRWQygFwAmdZplW8bNxER0ajg6BiG6QDalVL/BeBFALEAqgEEikiitY+HiNxrvb0CwCQA9wPIEZEJdnb7OYAQJ+MLBaC37vt3IpLgxGP8AdRbb//Ypv0EgA02uU0E8AmA74lIiLXNW0RCnYj5LQA5Sqnyvh2le6hiKoAaJ2IlIiIaFRxNSUQBeFFELAC6ADyplOoUkSwAr4qIv3Ufr4jIFQD/DuD7SqmLIvIbdJ/18OM++zwG4B8A7B7oia0HK3bYHEw4D8AlJ3LaBuCAiDQB+ADAHGv7L9B9cKUegBnAz5VS/yMijwPYJyLjrf1+BuCL/mK2FkoPAZgpIqtt+pxQSm1B9xTFJ9ZjOoiIaITpY//O3lTxtxZ56mOnLgS1bt26GZMnT+564YUXrgCAxWKBv79/TH19fUVubu6k8vJy77y8vNq2tjZZtGhRaFFRUbVGo0Ftba3Hhg0bZtbU1Ixva2tznzJlSmdxcXHfzyGcOnXKc+nSpXMvXLhQ6e7uDpPJhKSkpLm1tbWe586d048fP/7GVHdWVlaQXq/36rl/9epVj66uLrly5Ypu4cKFoUVFRdUeHh639Do4mpJ4H8D7dtpPo3sUoa9Qmz6v9rPPv1rPoJhgPX0SSqkPAXzYp2s2uof9ISJ+AM4qpers7O+BPvffBfCunX5tuLl4gVLqAwDx9mLtJ+YidE9l9OdHAF4faH9ERDT2VFVVaVNSUlp67p85c2bcpEmTTH5+fpbKykqvqKiodgDIyckJyMjIaNJouj+CV61aNWfNmjWNa9eubQKAkpISrb39x8bGGoODg435+fn+q1evbt64cePdISEhxuDg4L+dOHHCZ8mSJa09fQ8ePFhjG0dSUlL4jh07ajw9PVVSUlLL7t27Jz355JO3dFVKV13pcTOAWQ76XEf3mRNQSrUopR4Z9qgG5kzMAKBXSv1puIMhIqLbS3V1tTYuLu7GypTWlSo7gO5iIiYmpgMA9u/ff9fKlSuvA4DJZEJJSYlvamrqjQ/7hISEDgBISkoKqamp6TUMsGnTpiu5ubmT9+zZM6G4uNhn586ddenp6c3Hjx/3sxfT5cuXNWlpaaHPPPPMpdWrVzcDQFZW1vX8/PxJt5qfSwoGpVSxUqrCQZ+3b6dhfWditvbbNRLxEBHR7aOxsdHdZDLJrFmzbnxu6XQ6bWRkZDsAnD17VhsfH99hNBrl4sWL48PCwjoBQKPRIDExsWX+/Pn3Zmdnzy4oKPDueXxhYaEhKCioy/Z5VqxY0dLQ0OCxbdu2Ge+888658ePHq4ceeqj15MmTvn1jam1tdVu8eHFIZmbmtS1btlztaY+Pj++oqKjw7tvfEa4lQURENEilpaXasLCwXmfH6fV6bXR0dIfBYPDQarWWgIAAc0NDg8bX17fXl+HCwsKzeXl55/z8/MzLly8P3bt3r70TBm6Ii4v7Zv369Vd6ihNfX1+Ln5+f2XY0wmQyITMz8zshISHGHTt29Dr+T6PRwMPDQzU1Nd1SDcCCgYiIaJCqqqo8g4KC/tZz32w2o6yszCclJaXVOjXRDgDe3t6Wzs7OXp+9bm5uSE1NbXvzzTfrli5dek2n09k9hqFHdXW154IFC9pt25KTk1sOHz58Y1risccem20ymWTfvn019vbR1dUlXl5et3Q9IBYMREREgzRnzpzO06dPe7W3twsAPPfcc9MSEhLapk6datbpdNqIiIgOAAgMDDSbzWbp6Xfo0CE/o9EoAFBfX68pKSnxSUtLawGAxMTE0PPnz990KoPBYNDGxcX1Gs3IyMhoLigo8AeAzZs3T6uoqPA6evToOXtnQjQ0NLhPmDDBZHtWhTOGZbEmIiIiV3H2NMihtGLFipb33nvPb+7cuZEeHh4qNja2bc+ePbVA99REenp6c0/f+++/v7mgoMBn2bJlrQcOHJi4adOmWV5eXpZx48aprVu3XkpOTv7GbDajtrZ2fGBgYK/pC4PB4OHr62v29/fvdbHD2NhY45dffun52WefjX/55ZenT58+vfO+++4L79keHBxsPHbs2JcA8N577/klJyc34xaxYCAiIhoCu3btqgNw0+n/R44cOW97f+PGjY3bt2+fsmzZstb8/Pxae/s6deqUZ3p6epOPj0+vUYCQkJCuurq6SnuPiY2Nbauvr9copQYsmPbt2zdp+/btN8XpCAsGF4mcOBmlWU+5OozbF18bIhqjFi5c2F5aWtpiMpnQcy2GvuLj443x8fG39KGel5d3wVEfo9EoGRkZ1+fNm/c3R337EqW4BpIriEgrui+zfacKAHDVYa+xi/nfufnfybkDg89/tlIq0LZBp9PVREdH38mv6ZDR6XQB0dHRQfa2cYTBdaqVUnGuDsJVRKSU+TN/V8fhCndy7gDzH814lgQRERE5xIKBiIiIHGLB4Do7XR2AizH/O9udnP+dnDvA/EctFgwuopS6o//TMH/m7+oYXOVOzh1g/qMZCwYiIiJyiAXDMBORNBGpFhGDiDxrZ/t4Efm9dXuxiASNfJTDx4n8nxaRKhGpEJE/ichsV8Q5HBzlbtPvYRFRIjKmjhx3Jn8RWWl9/z8TkbyRjnE4OfG7P0tE/iwi5dbf/3RXxDkcROQtEflKRPT9bBcRedX62lSISOxIx0i3jqdVDiMRcQfwGoAH0X31r09F5IhSqsqm2xMAmpRSISLyKIBfAfjByEc79JzMvxxAnFKqXUSeBPBrjIH8ncwdIuIL4KcAikc+yuHjTP4iMhfAcwC+p5RqEpHJrol26Dn5/v8MwH6l1BsiEgHgOICgEQ92eOwB8BsAv+tn+2IAc60/9wF4w/rvkHg4Zs2CodoXABw6/bZTl5pet27djMmTJ3e98MILVwDAYrHA398/pr6+viI3N3dSeXm5d15eXm1bW5ssWrQotKioqFqj0aC2ttZjw4YNM2tqasa3tbW5T5kypbO4uPgLe8+RlZUVpNfrvXruX7161aOrq0vuvvvuv5WXl5/pWR/i1KlTnkuXLp174cKFSnd3d5hMJiQlJc1dtWrVtT179gQUFRVV21tnYiAcYRheCQAMSqkvlVKdAPIBZPbpkwngt9bbBwF8X0RkBGMcTg7zV0r9WSnVs+raJwBmjHCMw8WZ9x4A/gXdRaJxJIMbAc7kvxbAa0qpJgBQSn01wjEOJ2fyVwB6Vhf0B3AJY4RS6i8Arg3QJRPA71S3TwBMEJFpIxPd8KmqqtLGxMTcWBTqzJkz4yZNmmTy8/OzVFZWekVFRbUDQE5OTkBGRkZTz1UeV61aNSczM7NJr9d/XlNTo8/JybnY33McPHiw5syZM1VnzpypOnz4sMHd3V29/vrr5+Pi4r45ceKET0+/2NhYY3BwsDE/P98fADZu3Hh3SEiI8amnnvo6KSmpZffu3ZNuNT8WDMPrbgC2b3ydtc1uH6WUCUAzgLtGJLrh50z+tp4A8N6wRjRyHOZuHYadqZQ6NpKBjRBn3vtQAKEi8pGIfCIiaSMW3fBzJv9tAH4oInXoHl34x5EJ7bZwq38bRoXq6mptXFzcjWWnrctadwC9i4n9+/fftXLlyusAYDKZUFJS4puamtra87iEhIQOAEhKSgqpqamxOwxw+fJlTVpaWugzzzxzafXq1c3p6enNx48f97Pts2nTpiu5ubmT9+zZM6G4uNhn586ddQCQlZV1PT8/nwUDjU4i8kMAcQBedHUsI0FE3AC8DGCzq2NxIQ26h6QfALAKwC4RmeDSiEbWKgB7lFIzAKQD2Gv9vaBRqLGx0d1kMsmsWbNurC6p0+m0kZGR7QBw9uxZbXx8fIfRaJSLFy+ODwsL6wQAjUaDxMTElvnz59+bnZ09u6CgwLvn8YWFhYagoKCuvs/V2trqtnjx4pDMzMxrW7ZsuQoADz30UOvJkyd9bfutWLGipaGhwWPbtm0z3nnnnXM90xXx8fEdFRUV3n336wh/OYdXPYCZNvdnWNvs9hERDbqHJr8ekeiGnzP5Q0SSATwPIEMpdcsLotymHOXuCyASwIciUgPguwCOjKEDH5157+sAHFFKdSmlzgP4At0FxFjgTP5PANgPAEqpIgCe6F5n4U7g1N+G0aS0tFQbFhbWYdum1+u10dHRHQaDwUOr1VoCAgLMDQ0NGl9f315LVhcWFp7Ny8s75+fnZ16+fHno3r17+y2cTSYTMjMzvxMSEmLcsWPHjWksX19fi5+fn7nviERcXNw369evv2JbyGg0Gnh4eKimpqZbqgFYMAyvTwHMFZE5IjIOwKMAjvTpcwTAj623swB8oMbOimAO8xeR+QBy0V0sjKU57AFzV0o1K6UClFJBSqkgdB+/kaGUKnVNuEPOmd/9w+geXYCIBKB7iuLLkQxyGDmT/wUA3wcAEbkH3QVD44hG6TpHADxmPVviuwCalVKXXR3UYFRVVXkGBQXd+MJjNptRVlbmk5KS0mqdmmgHAG9vb0tnZ2evz143Nzekpqa2vfnmm3VLly69ptPptP09z2OPPTbbZDLJvn37avpuS05Objl8+HCvaYnq6mrPBQsWtPft29XVJV5eXrf0WcOCYRhZj0l4CsD7AD5H9xHRn4nICyKSYe32nwDuEhEDgKcB9Hv63WjjZP4vAvABcEBETotI3z+qo5KTuY9ZTub/PoCvRaQKwJ8BbFFKjYnRNSfz3wxgrYjoAOwD8PhY+bIgIvsAFAEIE5E6EXlCRNaJyDprl+PoLg4NAHYBWO+iUIfMnDlzOk+fPu3V3t4uAPDcc89NS0hIaJs6dapZp9NpIyIiOgAgMDDQbDabpaffoUOH/IxGowBAfX29pqSkxCctLa0FABITE0PPnz9/Y8Rg8+bN0yoqKryOHj16zt4ZDhkZGc0FBQX+tm0Gg0EbFxfXa+SjoaHBfcKECaaeKQpncXlrIiIa1W6X5a3Xrl074/jx4xM9PDxUbGxsW25u7sXAwEBzRkbGnPT09OZ169ZdA4CVK1fOzs7OvrZs2bLWRx99dPZHH33k6+XlZRk3bpx6+umnG9asWdNkNpsxc+bMqC+++ELv4+Ojqqurx4WHh0dNnz6909fX19zznMHBwcZjx47dGJkLDw+P0Ov1VRqNBgaDweOBBx4Ir6urq7SN8+2335748ccfe+/atauubw4DLW/NgoGIiEa126VgcNbJkye9tm/fPuXw4cPn++vz6aefeubm5gbs3r37pg/1gWRnZ896/PHHv05JSfmmvz4pKSnB27dvr5s3b95Nx4wNVDDwwk1EREQjaOHChe2lpaUtJpMJPddi6Cs+Pt4YHx9/S8UCAOTl5V0YaLvRaJSMjIzr9ooFRzjCQEREo9poG2G4nQ00wsCDHomIiMghFgxERETkEAsGIiIa7SwWi2WsrMHjMtbX0NLfdhYMREQ02ukbGxv9WTR8exaLRRobG/0B2F2SHOBZEkRENMqZTKafNDQ07G5oaIgEvwh/WxYAepPJ9JP+OvAsCSIiInKIlRgRERE5xIKBiIiIHGLBQERERA6xYCAiIiKHWDAQERGRQ/8HWgXGkkqxQH4AAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "\"\"\"\n", + "define a auxiliary function which takes an array of\n", + "\n", + " {\n", + " \"variables\": [\"variable_0\", \"variable_1\"],\n", + " \"metrics\": {\n", + " \"mi\": ...,\n", + " \"si\": ..., \n", + " \"ci\": ...,\n", + " \"ui_0\": ...,\n", + " \"ui_1\": ...\n", + " }\n", + " },\n", + "\n", + "\"\"\"\n", + "\n", + "\n", + "def plot(data):\n", + " \n", + " # definte the size of figure\n", + " plt.figure(figsize=(6, 0.5*len(data)))\n", + " \n", + " labels = []\n", + " metrics = []\n", + " \n", + " # suffix each variable name with an alias {Y, Z}\n", + " for v in data:\n", + " labels.append(\", \".join(map(lambda p: \"%s (%s)\" % p, zip(v['variables'], ['Y', 'Z']))))\n", + " \n", + " for m in metric_keys:\n", + " mm = []\n", + " # extract corresponding metric from elements in array\n", + " for v in data:\n", + " mm.append(v['metrics'][m]/v['metrics']['mi'])\n", + " metrics.append(mm)\n", + " \n", + "\n", + " left = np.array([0]*len(labels))\n", + " \n", + " # plotting\n", + " for i, kk in enumerate(metric_keys):\n", + " plt.barh(\n", + " labels, metrics[i], align='center', height=.5, left=left, label=latex_labels[i],color=colors[i],\n", + " )\n", + " left = left + metrics[i]\n", + " plt.legend(loc='center left', bbox_to_anchor=(1, 0.5))\n", + "\n", + "plot([\n", + " {\n", + " \"variables\": [\"sex\", \"race\"],\n", + " \"metrics\": {\n", + " \"mi\": dit.shannon.mutual_information(q_SGR, 'S', 'RG'),\n", + " \"si\": si_SGR, \n", + " \"ci\": ci_SGR,\n", + " \"ui_0\": ui_SG_R,\n", + " \"ui_1\": ui_SR_G,\n", + " }\n", + " },\n", + "])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "[note: writing some summary here]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Other variable pairs\n", + "\n", + "In the following, we are going to compute the information decomposition between `income` and several pairs of variables, namely\n", + "\n", + "- education and sex\n", + "- education and race\n", + "- race and occupation\n", + "- **education and occupation [note: didn't converged]**\n", + "- age-group and sex\n", + "- hours-per-week-group and occupation.\n", + "\n", + "We first define a function that computes the decomposition for us." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [], + "source": [ + "def information_decomposition(dist, src, to=\"\"):\n", + " rvs = src+to\n", + " \n", + " P = dist.marginal(rvs)\n", + " \n", + " variables = P._rvs\n", + " \n", + " q_SXY = admUI.computeQUI(distSXY = P)\n", + " \n", + " h_SgXY = dit.shannon.conditional_entropy(q_SXY, 'S', 'XY')\n", + " \n", + " ui_SX_Y = dit.shannon.conditional_entropy(q_SXY, 'S', 'Y') - h_SgXY\n", + " ui_SY_X = dit.shannon.conditional_entropy(q_SXY, 'S', 'X') - h_SgXY\n", + "\n", + " si_SXY_1 = dit.shannon.mutual_information(q_SXY, 'S', 'X') - ui_SX_Y\n", + " si_SXY_2 = dit.shannon.mutual_information(q_SXY, 'S', 'Y') - ui_SY_X\n", + " \n", + " # sanity check\n", + " assert math.isclose(si_SXY_1, si_SXY_2, abs_tol=1e-6), \"SI_S_XY: %f | %f\" % (si_SXY_1, si_SXY_2)\n", + "\n", + " si_SXY = si_SXY_1\n", + " \n", + " ci_SXY = si_SXY - dit.multivariate.coinformation(P, rvs)\n", + " i_S_XY = dit.shannon.mutual_information(P, 'S', to) \n", + "\n", + " # sanity check\n", + " assert math.isclose(i_S_XY, si_SXY + ci_SXY + ui_SX_Y + ui_SY_X, abs_tol=1e-6), \\\n", + " \"MI = decompose : %f | %f\" % (i_S_XY, si_SXY + ci_SXY + ui_SX_Y + ui_SY_X)\n", + " \n", + " uis = [ui_SX_Y, ui_SY_X]\n", + " return {\n", + " \"variables\": tuple(map(lambda x: rvs_to_name[x], to)),\n", + " \"metrics\": {\n", + " \"mi\": i_S_XY,\n", + " \"si\": si_SXY, \n", + " \"ci\": ci_SXY,\n", + " \"ui_0\": uis[variables[to[0]]-1] ,\n", + " \"ui_1\": uis[variables[to[1]]-1]\n", + " }\n", + " }\n", + "\n", + "decomp_S_HO = information_decomposition(dist_census, 'S', 'HO')" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [], + "source": [ + "decomp_S_EG = information_decomposition(dist_census, 'S', 'EG')" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [], + "source": [ + "decomp_S_ER = information_decomposition(dist_census, 'S', 'ER')" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": {}, + "outputs": [], + "source": [ + "decomp_S_RO = information_decomposition(dist_census, 'S', 'RO')" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": {}, + "outputs": [], + "source": [ + "decomp_S_AG = information_decomposition(dist_census, 'S', 'AG')" + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": {}, + "outputs": [], + "source": [ + "# didn't converge, can we remove it?\n", + "# decomp_S_EO = compute_decomposition_from(dist_census, 'S', ['E', 'O'])" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAooAAACnCAYAAACW2X1NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de5zVVb3/8dcbBmW4DIog4gVREAhBEAbMNMkkUVJAQ80s0zp1LM1jqaUnLbNfF9NjGWbejmIWkuIlzFQ8mmim4HAZGBEEdVBJFBO5CAgz8/n9sdfGzXYPe9Rhhsv7+Xjsx3y/67u+a32+azb6mbW+370VEZiZmZmZ5WvR3AGYmZmZ2dbJiaKZmZmZFeRE0czMzMwKcqJoZmZmZgU5UTQzMzOzgpwompmZmVlBJc0dgNmOplOnTtG9e/fmDsPMbJsyY8aMtyKic3PHsaNxomjWxLp3705FRUVzh2Fmtk2RtLi5Y9gReenZzMzMzApyomhmZmZmBTlRNDMzM7OCnCiamZmZWUF+mMWsiVUtf5M+k65t7jB2KPMn/bS5Q9jhVL3Qo7lDsO3Ej+t6c/fsW5s7jB2WZxTNzMzMrCAnimZmZmZWkBNFMzMzMyvIieJ2RtIZkhr1BjhJYyT1zdm/XNLwRmr7YEn/K+lASS9IKs059oCkUyUdJ+nyxujvI8QnSY9JKpN0gqTZea86ScdK6izpoeaI0czMbEtxomgNMQbYmChGxI8i4v8aqe3/Bn4bEc8B9wA/hExyCrSKiDuAB4DjJbVppD4/jJFAZUSsjIh7I2Jg9gVcBzwJPBwRy4DXJR3WDDGamZltEU4UtyGSvixpeprJukFSy1R+ZpqNmw4cllN/vKSxOfurc7Z/IGmupEpJv0xl35D0bCq7W1IbSZ8CRgFXpn575LYr6ShJs1Jbt0jaOZVXS/qJpJnpWJ8C19MeOCgiKlPR5cBJkgYCvwTOBoiIAB4HjisyPgfmjM8cSQfUN26S9pW0UFInSS0kPSnp6ALNngb8pUBfvYAfAV+JiLpUfF+qb2Zmtl1woriNkPQJ4BTgsDSbVQucJqkr8BMyCeLh5Mz8baatY4HRwCERMQD4VTp0T0QMSWXPA1+PiH8Ck4EL00zaiznttAbGA6dERH8yH7f0rZyu3oqIQcDvgQsKhFIOVGV3ImJNqvcEMDEiFubUrQA+XeTSzgKuSeNTDrxW37hFxGLgihTb+cC8iJhSoM3DgBm5BZJaAROA8yPilYbEKOmbkiokVdSuXF2oipmZ2VbHieK24yhgMPCspNlpf3/gEODxiFgWEeuBPzegreHArSkxIyLeTuX90szaXDIzYwcWaac38HJEvJD2bwOOyDl+T/o5A+he4PyuwLLcgoi4H3iHzLJurjeBPYvE8zTw35J+AOwbEWupf9yIiJuBMjIJZqFEFqBjRKzKK/sp8FxE5I91vTFGxI0RUR4R5S3L2hW5DDMzs62DP3B72yHgtoi4eJPCzL189akh/TEgqQWwU5E+xgNjIqJS0hnAZz5qsMl76Wcthd9ra4HWBcrr0itX61S/XhExQdI04PPA3yT9J/WMG0C653HvtNsOyE8IAWoktcguL0v6DPAFYFCBukVjNDMz25Z4RnHb8SgwVtLuAJI6StoXmAYMk7RbWhI9KeecajKzaZC5z7BV2n4EODP7cIikjqm8PZkHMlqx6b12q9KxfAuA7pJ6pv2vAFM/xDU9D/QsWiujF2mZOj19/Iv8CpL2B16KiN+Sua/wIOofN8gsPf+JzL2GN9XT7wLSDKSkXYFbgdMLzDJuEqOZmdn2wIniNiIi5gGXAFMkzSGT7HWNiNeBy8gsuz5FJvnKuolMElkJHAq8m9p6iMx9hxVpOTa77HopmcTzKWB+TjsTgQvTQysbv5crItYBZwJ3peXqOuD6D3FN84EO6aGWYo4k8/QzQA9gZYE6JwNV6Zr6AX+ob9wkDQOGAFdExJ+A9ZLOLNDmA7w/s3oWsDvw+7yPyDmlQIxmZmbbPGUeKDVrHpK+C6xK9wvWV6cLMCEijkr7fwS+mz6SZkvH15VMwvm5BtR9AhgdEcs3V691j27R/YrvN1aI1gD+ruem5+96tsaS/a5nSTMiory549nReEbRmtvvef9exvp0I/NkMgAR8eWmSBJTX68DN0kq21w9SZ2Bq4sliWZmZtsSzyiaNbHy8vKoqKho7jDMzLYpnlFsHp5RNDMzM7OCnCiamZmZWUFOFM3MzMysICeKZmZmZlaQE0UzMzMzK8iJopmZmZkV5ETRzMzMzApyomhmZmZmBTlRNDMzM7OCnCiamZmZWUElzR2AmZmZ2UcxY8aM3UtKSm4G+uHJr4+qDqiqqan5j8GDB7+Zf9CJopmZmW2TSkpKbt5jjz0+0blz5+UtWrSI5o5nW1RXV6dly5b1Xbp06c3AqPzjzr7NzMxsW9Wvc+fOK50kfnQtWrSIzp07ryAzK/sBnlE0a2JVy9+kz6Rrm6y/+ZN+2qjtVb3Qo1HbMzMr5sd1ves71MJJ4seXxrDg5KFnFM3MzMysICeKZmZmZlaQE8VtmKQzJDXqGqakMZL65uxfLml4I7V9sKT/lXSgpBckleYce0DSqZKOk3R5Y/TXGLIxp+1bJc3OeVVLeiMdO0fS15o3WjMzs8blexQt3xjgr8A8gIj4USO2/d/A/4uI5yTdA/wQuETSGKBVRNwhScBPJf0yItY0pFFJJRFR04hxfiBmgIg4M6fPFsDjwB9S0S3AU+mnmZk1gz6Trh3cmO3NH3vOjGJ1Fi9e3Orss8/ep7q6eufVq1e37NKly/pp06a9cOWVV3aaNWtW2wkTJixuzJiammcUt1KSvixpepq5ukFSy1R+ZpqNmw4cllN/vKSxOfurc7Z/IGmupEpJv0xl35D0bCq7W1IbSZ8i82j8lanfHrntSjpK0qzU1i2Sdk7l1ZJ+ImlmOtanwPW0Bw6KiMpUdDlwkqSBwC+BswEiIsgkYMcVGZ8zJE2W9BjwqKR2kh7NiWF0Tt3TJc1J13p7KuucrvvZ9DqsQB/5Mef6b2BZRNyc4l4DVEsaurm4zcxs+3LqqafuN3r06OVVVVXPV1dXV40bN+5VgLlz57bp379/gyY8tmZOFLdCkj4BnAIcFhEDgVrgNEldgZ+QSRAPB/rW38rGto4FRgOHRMQA4Ffp0D0RMSSVPQ98PSL+CUwGLoyIgRHxYk47rYHxwCkR0Z/MbPS3crp6KyIGAb8HLigQSjlQld1JidUFwBPAxIhYmFO3Avh0sWsDBgFjI2IYsA44IcVwJPA/yjgQuAT4bLrW/0rnXgP8OiKGAF8Abi4Wc1ZKBv8D+EbeoXrjlvRNSRWSKmpXri5UxczMtjE1NTVMnz69/YgRI1Zly4YOHboWYN68eaUDBw5c23zRNQ4niluno4DBwLOSZqf9/YFDgMcjYllErAf+3IC2hgO3ZpdxI+LtVN5P0pOS5gKnAQcWaac38HJEvJD2bwOOyDl+T/o5A+he4PyuwLLcgoi4H3gHuC6v7pvAnkXiAXgk53oE/FzSHOD/gL2ALsBngbsi4q3UZ7b+cODaNL6TgTJJ7YrFnOr8kUxi/XZe/XrjjogbI6I8IspbluV3Y2Zm26KSkhIOPfTQlQcffPCBX/rSl/adMmVK2+yxhQsXlg4ZMsSJom0RAm5Ls3oDI6J3RFxW5Jwa0u8z3T+3U5H644Fz0uzgT4DWHy9k3ks/ayl87+vaevqoS69crVP9Yt7N2T4N6AwMTrOwb9TTX1YL4JM5Y7xXRORP9RWKeRzwl4h4tECbDY3bzMy2E1OnTl04YcKEF8vKympPOOGEXrfffvsuixYtalVaWlrXqVOn2uaO7+Nyorh1ehQYK2l3AEkdJe0LTAOGSdpNUivgpJxzqsnMQkLmPsNWafsR4ExJbbJtpfL2wOupndNy2lmVjuVbAHSX1DPtfwWY+iGu6XmgZ9FaGb1IS76STpD0iwac0wF4MyI2SDoS2DeVP0bmXsjdUnvZ658CfCd7crpXcrMxp3s1B5B5CGezcZuZ2Y6hRYsWjBgxYvX111//2vHHH/92ZWVl6YwZM9r06dNnm78/EZwobpUiYh6Z++qmpKXUR4CuEfE6cBnwNJknbJ/POe0mMklkJXAoabYtIh4is7RakZZZs/cPXkom8XwKmJ/TzkTgwvTQysav4IiIdcCZwF1puboOuP5DXNN8oEN6QKSYI4EH0nYPYGUDzvkTUJ5iO510TRHxHPAzYGoam6tT/XNT/TmS5gFnNSDmn5GZtcw+ZJR9ZT/m5zAyvyszM9sB3H333WXr1q0TwJIlS0qmT5/e7phjjllZWVlZ2rdv3+1ihUmZh0zNtjxJ3wVWZZ8UrqdOF2BCRByV9v8IfDciltV3zpbUkJhTvYOB70XEV4q12bpHt+h+xfcbK8Si/BV+Zrat+3Fdb+6pHD8jIspzyysrK6sHDBjwVnPF9cUvfnHfp556qn2bNm3qdtppp/je97639Mwzz1w+atSo/UaOHLnirLPOyr+XfatVWVnZacCAAd3zy/05itaUfs+my+WFdAPOz+5ExJe3aETFNSRmgE5kZmnNzGwHMXHixIKfkTh58uSXmzqWLcWJojWZtHx9e5E6zzZROA3SkJhTvQYvOffbdXcqxp7zseL6UBq5r36N2pqZWXF3A9L45g5jh+R7FM3MzMysICeKZmZmZlaQE0UzMzMzK8iJopmZmZkV5ETRzMzMzApyomhmZmZmBTlRNDMzM7OC/DmKZmZmtn34YpfBjdrexDdmNKTaLbfcsutvfvObLuvXr29RU1Oj0aNHv73HHntsmDVrVtsJEyYU/FDubYVnFM3MzMw+onHjxu129dVX73Hvvfe+OH/+/HmzZs16vmPHjrVz585t079//zXNHd/H5UTRzMzM7CN4++23W1x66aX73HnnnS/26NFjA0D79u3rfvjDH745b9680oEDB65t7hg/LieKZmZmZh/BhAkTdh0wYMDqvn37rs8/tnDhwtIhQ4Y4UTQzMzPbEVVVVZX279//A8ngokWLWpWWltZ16tSptjniakxOFM3MzMw+grZt29bW1dV9oHzGjBlt+vTps83fnwh+6tmsyVUtf5M+k65tsv7mT/ppo7b3hfkjG7U9M7Ni7p59a3OHUNBxxx234vTTT9//4osvfmOfffapWbt2ra677rrd/v3vf5f07dt3m192BieKZmZmtr1o4MfZNJYjjzxyzUUXXfSvz33uc73q6uqoqanRSSed9O/nn3++dOTIkSuaMpYtxYmimZmZ2Ud09tlnv3322We/3dxxbCm+R3EHIalU0lRJbSXNl9Q/59iFkm6Q1FnSQ80Z50chaaCkkTn7oyRd1EhtZ8etpaSzJc3OeVVJCkmfkNRf0vjG6NPMzGxr4URxK6KMLfU7+RpwT0S8C5wHXJf62ws4C7goIpYBr0s6bAvFsKUMBDYmihExOSJ+2UhtZ8etNiJ+FxEDsy9gMvCniHg+IuYCe0vq1kj9mpmZNTsnis1MUndJCyT9AagC9pH0e0kVkp6T9JOcukMk/VNSpaTpktqnma4rJT0raY6k/6ynq9OAvwBExEPA68DpwK+ByyJieap3X6pbLO7vpRm1Kknn5ZSfnuKolHR7Kusi6d5UVinpU+m6q3LOu0DSZWn7cUnX5MzaDU3lQyU9LWlWGofeknYCLgdOSfVPkXSGpGtzxvexFNOj2URO0nhJv03tvCRpbLFxy7v+I4CTgW/nFN8PfLHY2JmZmW0rfI/i1uEA4KsR8QyApB9GxNuSWgKPSjoImA/8GTglIp6VVAasBb4OrIiIIZJ2Bp6SNCUiXs42npKp/SOiOqfP84DpwMKIuD2nvAL4f5sLVtJg4EzgEEDANElTgfXAJcCnIuItSR3TKb8FpkbECema2gG7FhmTNhExMCVktwD90hh8OiJqJA0Hfh4RX5D0I6A8Is5J8Z2R08444LaIuE3S11IsY9KxrsDhQB8ys4OT8q6z0LghaRdgPPCViFiZc6gCuAj4VZFrMzMz2yY4Udw6LM4micnJkr5J5vfTFegLBPB6RDwLkE1QJB0NHJQzI9aBTOL5ck57nYB3cjuMiH9Jegz4a14sbwJ7Fon3cODetIyNpHuAT6cY74qIt1If2Zt7P0tm9pKIqAVWSCqWKN6R6j8hqSwlZ+2B2yQdkPpqVaQNgEOBE9P27WyaxN0XEXXAPEldCpz7gXFLrgduj4in8srrHbv0+/wmQEmnYpduZma2dXCiuHV4N7shaT/gAmBIRCxPD0i03sy5Ar4TEQ9vps7aetqoS69crVP9La2GTW99yI8vCuz/FPh7mpnsDjz+MWN4L2dbBY5/YNwkfRXYF/hygfr1jl1E3AjcCNC6R7f8azMzM9sq+R7FrU8ZmcRxRZrlOjaVLwC6ShoCkO5PLAEeBr4lqVUq7yWpbW6D6f7DlpI2l3Bm9SJzrySS9pL0aIE6TwJjJLVJfZ2Qyh4DTpK0Wzo/u/T8KPCtVNZSUgfgDWB3SbulJfPj8vo4JdU/nMzS+goys6VL0vEzcuquIjPbWMg/ef++wdNSnA2SP26S9gd+DpwWETUFTtk4dmZmZtsDJ4pbmYioBGaRuR9vAvBUKl9PJnkaJ6kSeITMDNbNwDxgZno45AYKzxRPIbNkXMyRwANpuyuZmb/8GGeSuUdvOjANuDkiZkXEc8DPgKkpxqvTKf8FHClpLjAD6BsRG8g8hDI9Xcv8vG7WSZpFZpn366nsV8AvUnnuNf4d6Jt9mCWvne8AZ0qaA3wlxfJh5I7bD4A2wD3a9GNyPp2O546dmZnZNk8RXgXbEUgaBHw3Ir5SpN4TwOi07H0O8EpETG6SIN+P4XHggoioaMp+64mloeO2MzAVOLye2caNWvfoFt2v+H4jRrl5/go/M9vW3T37ViTNiIjy3PLKysrqAQMGvJXdrxr0qcGN2W+/mf8s+k0vZ5111t677777hssvv/wNgLq6Ojp06DBwyZIlc2644YaOs2bNajthwoTFq1ev1pFHHtnr6aefXlBSUsLixYtbnX322ftUV1fvvHr16pZdunRZP23atBfy2585c2br448//oBXXnllbsuWLampqWHYsGEHLF68uPWLL75YtfPOO29M5MaOHdu9qqqqTXb/rbfearVhwwa98cYblYcffnivp59+ekGrVoVv76+srOw0YMCA7vnlnlHcQaRZwL+np44LktQZuDr7UTkRcW1TJ4lbm4aMW9KNzGdRbjZJNDOz7cu8efNKBw4cuPH+9Pnz5+/UsWPHmrKysrq5c+e26d+//xqAcePGdRo1atTykpLMgtipp5663+jRo5dXVVU9X11dXTVu3LhXC7U/aNCgdT169Fg3ceLEDgDnnnvuXj179lz32c9+dsUjjzzSLrfupEmTqufPnz9v/vz58+67775FLVu2jOuuu+7l1q1bx7Bhw1befPPNHQv1sTlOFHcgEXFLeuq4vuPLIuK+poypnjg+szXMJmYVG7dUZ2FEPN5EIZmZ2VZiwYIFpeXl5Wuy+zNmzGjTp0+ftbBpEnnnnXfudvLJJ78DUFNTw/Tp09uPGDFiVfa8oUOHrgUYNmxYz+rq6k2m/c4777w3brjhht3Hjx+/y7Rp09rdeOONr40cOXLF3/72t7JCMb3++uslxxxzTK8LLrjgX6eddtoKgLFjx74zceLED50o+qlnsybWb9fdqRh7TtN12Mh93d2orZmZbbuWLVvWsqamRt26ddu4mlRZWVnar1+/NQALFy4sHTJkyNp169bp1Vdf3bl3797rAUpKSjj00ENXHnzwwQeOGDHinTPOOOOto48++l2AqVOnLsrv58QTT1x50UUX7X3ZZZft/cQTT8zfeeed4/Of//yqn//8513z665atarFscce23P06NFvX3jhhRuX5YcMGbJ2zpw5bfPrF+MZRTMzM7OPoKKiorR3796bfCxaVVVV6YABA9YuWrSoVWlpaV2nTp1qly5dWtK+fftNbk2aOnXqwgkTJrxYVlZWe8IJJ/S6/fbbd9lcX+Xl5e9++9vffiOblLZv376urKysNnf2saamhtGjR+/fs2fPdddcc82/cs8vKSmhVatWsXz58g+V+zlRNDMzM/sI5s2b17p79+4bP5O3traWGTNmtDv66KNXpSXoNQBt27atW79+/SY5V4sWLRgxYsTq66+//rXjjz/+7crKytLN9bVgwYLWgwcPXpNbNnz48JX33XffxuXn008/fd+amhrdcccd1YXa2LBhg9q0afOhnmJ2omhmZmb2Eey3337rZ8+e3WbNmjUCuPjii7sOHTp09R577FFbWVlZ2rdv37UAnTt3rq2trVW23t133122bt06ASxZsqRk+vTp7Y455piVAIceemivl19++QOPJi9atKi0vLx8k9nLUaNGrZgyZUoHgPPPP7/rnDlz2tx///0vFnqyeenSpS132WWXmtynpBvC9yiamZnZdqEhH2fTmE488cSVDz74YNkBBxzQr1WrVjFo0KDV48ePXwyZJeiRI0euyNY94ogjVkyZMqXdmDFjVt111127nnfeed3atGlTt9NOO8Wll176r+HDh79bW1vL4sWLd+7cufMmy9SLFi1q1b59+9oOHTps8m1qgwYNWvfSSy+1fu6553a++uqr99xzzz3XH3LIIX2yx3v06LHugQceeAngwQcfLBs+fPgKPiQnimZmZmYf0U033fQa8Fp++eTJk1/O3T/33HOXXXXVVV3GjBmzauLEiYsLtTVz5szWI0eOXN6uXbtNZv169uy54bXXXptb6JxBgwatXrJkSUlEbDZJvuOOOzpeddVVH4izGCeKZmZmZlvY4YcfvqaiomJlTU0N2c9SzDdkyJB1Q4YM+VDJ3IQJE14pVmfdunUaNWrUOwcddNB7xermc6JoZmZm1gTOO++8fzdHv61bt45zzjnnI/Xth1nMzMzMrCAnimZmZmZWkBNFMzMzMyvIiaKZmZmZFeRE0czMzMwK8lPPZk2savmb9Jl0bXOHYR/C/Ek/be4QdihVL/Ro7hBsK9Nv5j+bO4QdlmcUzczMzKwgzyiamZnZduELA88c3Jjt3T371qJfCXjWWWftvfvuu2+4/PLL3wCoq6ujQ4cOA5csWTLnhhtu6Dhr1qy2EyZMWLx69WodeeSRvZ5++ukFJSUlLF68uNXZZ5+9T3V19c6rV69u2aVLl/XTpk17oVAfY8eO7V5VVdUmu//WW2+12rBhg/baa6/3Zs2aNT/7/c0zZ85sffzxxx/wyiuvzG3ZsiU1NTUMGzbsgFNPPfXt8ePHd3r66acXFPoe6M3xjKKZmZnZRzRv3rzSgQMHrs3uz58/f6eOHTvWlJWV1c2dO7dN//791wCMGzeu06hRo5Znv5Xl1FNP3W/06NHLq6qqnq+urq4aN27cq/X1MWnSpOr58+fPmz9//rz77rtvUcuWLeO66657uby8/N1HHnmkXbbeoEGD1vXo0WPdxIkTOwCce+65e/Xs2XPdOeec8+9hw4atvPnmmzt+2OtzomjNStJ5kk6X9A1Jf84pL5P0oqT9JV0l6bPNFF9XSX9N2z+TNDvn9YKkWkntJB0n6fLmiNHMzJrPggULSsvLy9dk92fMmNGmT58+a2HTJPLOO+/c7eSTT34HoKamhunTp7cfMWLEqux5Q4cOXQswbNiwntXV1QWn/V5//fWSY445ptcFF1zwr9NOO23FyJEjV/ztb38ry61z3nnnvXHDDTfsPn78+F2mTZvW7sYbb3wNYOzYse9MnDjRiaI1Lklb7PaE1PbXgAnAzcA+koanw5cDt0TES8A44KItFUcR3wNuAoiIH0bEwOwLeBb4RUSsBh4AjpfUZjNtmZnZdmTZsmUta2pq1K1bt5psWWVlZWm/fv3WACxcuLB0yJAha9etW6dXX3115969e68HKCkp4dBDD1158MEHH/ilL31p3ylTprTNnj916tRF3bt335Df16pVq1oce+yxPUePHv32hRde+BbA5z//+VX/+Mc/2ufWO/HEE1cuXbq01WWXXbb3vffe+2J2WXrIkCFr58yZ0za/3WKcKG4DJN0naYak5yR9M6f862lWa7qkmyRdm8o7S7pb0rPpdVg97Y6UND+1/ducmbPLJN0u6SngdkmtJd0qaa6kWZKOTPXOyPaZ9v8q6TNpe7WkX6eYH5XUuUAInwVmRkRNRARwFvAbSeXAUcCVABGxGNhN0h5FxukkSVWSKiU9kcpaSroyjcMcSf+Zyk9IcSnNGr5QT/tfAB4q0NeXgZ7AZSnGAB4HjttcjGZmtv2oqKgo7d2799rcsqqqqtIBAwasXbRoUavS0tK6Tp061S5durSkffv2Nbn1pk6dunDChAkvlpWV1Z5wwgm9br/99l3q66empobRo0fv37Nnz3XXXHPNv7Ll7du3rysrK6vNn4EsLy9/99vf/vYbuQlsSUkJrVq1iuXLl3+o3M+J4rbhaxExGCgHzpW0m6Q9gUuBTwKHAX1y6l8D/DoihpBJdG7Ob1BSa+AG4NjUdn4i1xcYHhGnAmeTyYX6A6cCt6XzN6ctUBERBwJTgR8XqHMYsPFG4YiYAzwMPAp8JyLW59Sdmepvzo+AERExABiVyr4OrEhjMQT4hqT9IuJe4PV0bTcBP46IpbmNSdoPWB4R7+WVdwd+CZwWEbn/8CuATxcKTNI3JVVIqqhdubrIZZiZ2bZg3rx5rbt3777x/xG1tbXMmDGj3dFHH70qLUGvAWjbtm3d+vXrN8m5WrRowYgRI1Zff/31rx1//PFvV1ZWltbXz+mnn75vTU2N7rjjjur8Y8OHD1953333bbL8vGDBgtaDBw9ek193w4YNatOmTXyYa3SiuG04V1Il8AywD3AAMBSYGhFvR8QG4K6c+sOBayXNBiYDZZLa5bXZB3gpIl5O+3fkHZ8cEdm/kg4H/ggQEfOBxUCvIjHXAdl7Dv+Y2sjXFViWV/Y7YElEPJ5X/iawZ5E+nwLGS/oG0DKVHQ2cnsZiGrAbmfED+A5wMfBeRORff8H4JLVM13NpRCxqaIwRcWNElEdEecuy/F+FmZlti/bbb7/1s2fPbrNmzRoBXCgxuScAABGzSURBVHzxxV2HDh26eo899qitrKws7du371qAzp0719bW1ipb7+677y5bt26dAJYsWVIyffr0dsccc8xKgEMPPbTXyy+/vHGG8Pzzz+86Z86cNvfff/+LhZ5YHjVq1IopU6Z0yC1btGhRaXl5+SYznUuXLm25yy671GSXohvKH4+zlUtLucOBQyNijaTHgWKzeS2AT0bEury2Hga6kJn5KvaJz+82ILwaNv1jY3NxFXpjri1wTl165Wud6tffQcRZkg4BPg/MkDQYEJnZyYcLnLJ36quLpBYRkd9vofguAV6PiFs/SoxmZrblNOTjbBrTiSeeuPLBBx8sO+CAA/q1atUqBg0atHr8+PGLIbMEPXLkyBXZukccccSKKVOmtBszZsyqu+66a9fzzjuvW5s2bep22mmnuPTSS/81fPjwd2tra1m8ePHOnTt3rgFYsGDBTldfffWee+655/pDDjlk48phjx491j3wwAMvQeZJ55deeql1TU0NJSUlLFq0qFX79u1rO3TosMn/0x588MGy4cOHr+BDcqK49etAZvlzjaQ+ZJaaIfMgxW8k7QqsIrPEPDcdm0JmtuxKAEkDI2J2RIzINiqpFNhfUveIqAZO2UwMTwKnAY9J6gV0AxYAZcC3JbUA9iIzy5nVAhgLTAS+BPyjQLvPk7nPryF6kWZNJf0CmJ6WjzeS1CMipgHTJB1LZvb1YeBbkh6LiA0p/iXAe8AtZJbSv0rmoZWr8vp8Aeie0/4ngTOAQZuJsaqB12NmZtuBm2666TXgtfzyyZMnv5y7f+655y676qqruowZM2bVxIkTFxdqa+bMma1Hjhy5vF27dgHQu3fv9RFRNPkdNGjQ6scee6zt0Ucf/W7Pnj03vPbaa3Pz69xxxx0dr7rqqg/EWYwTxa3fQ8BZkp4nk5w9AxARSyT9HJgOvA3MB7J/KZwL/E7SHDK/4yfIPCiyUUSslfRt4CFJ75JJPOtzHfB7SXPJzCKeERHvpYddXgbmkUn6Zuac8y4wVNIlZJZkCyWiDwK3FxsASa3IJJQVqag/mSX1fFdKOoDMLOKjQCUwh0yyN1OSyCwljwHOB56MiH+kZf1nJT0QEc9nG4uId9NH9PRMy8w/AdoAf880tdEXIuJF4EgyS9lmZmabOPzww9dUVFSszM78FTJkyJB1Q4YM+dDJ3IQJE17Z3PF169Zp1KhR7xx00EHvba5eIco8rGnbIkntImJ1+piZe8l8nMy9xc4rcL7I3Bu4MCJ+3UixrY6IojfjSboX+H5ELNxMnROAQRFxadp/OHd2dEtKfQ+OiEuK1OsCTIiIo4q12bpHt+h+xfcbK0RrAv6u56bl73q2fP1m/hNJMyKiPLe8srKyesCAAW81V1zbk8rKyk4DBgzonl/uh1m2bZelhzSqyMzs3fchz/9GOv85MkvcNzRyfA1xEZmHRjanBPif7E5TJYmpr3uB6gZU7UZmltLMzGy74RlFsyZWXl4eFRUVxSuamdlG9cwovtS/f//lLVq0cDLzMdTV1Wnu3Lm7DhgwYP/8Y55RNDMzs21V1bJlyzrU1dWpeFUrpK6uTsuWLetAPQ9j+mEWMzMz2ybV1NT8x9KlS29eunRpPzz59VHVAVU1NTX/UeigE0UzMzPbJg0ePPhN3v8mLtsCnH2bmZmZWUFOFM3MzMysICeKZmZmZlaQE0UzMzMzK8iJopmZmZkV5ETRzMzMzApyomhmZmZmBTlRNDMzM7OCnCiamZmZWUFOFM3MzMysIH+Fn1kTq1r+Jn0mXdvcYVgjmz/pp80dQrP6wvyRzR2Cbcfunn1rc4eww/KMopmZmZkV5ETRzMzMzArabKIoqbukqqYKZlskqVpSp+aOoxhJXSX9VdLuKeY9co79TtLFkvpLGt+MYX4kkj4j6VM5+2dJOr2R2u4q6a9p+2eSZue8XpBUK6mdpOMkXd4YfZqZmW0tmmVGUVKT3BspqWVT9NMYlLElfx/fA26KiDeBXwJXpX4HAZ8GroqIucDekrptwTi2hM8AGxPFiLg+Iv7QSG1/D7gptfvDiBiYfQHPAr+IiNXAA8Dxkto0Ur9mZmbNriGJSUtJN0l6TtIUSaUAkgZKekbSHEn3Sto1lT8uqTxtd5JUnbbPkDRZ0mPAo2mm5ok0M1Ml6dP5Hadz/pLaXCjpxznHvixpejr/hmxSKGm1pP+RVAkcmtfe7ySNStv3SrolbX9N0s+KtHu0pKclzZR0l6R2eW2XSnpQ0jcKXEdnSY+kMbxZ0uI0Nt0lLZD0B6AK2EfSqZLmpjG5IqeN1TnbY7Mzf5LGS7peUkWa4Tqunt/jF4CH0vaNQA9JRwK/A86JiA3p2P3AF+tpI9u/JF2ZYpwr6ZScYz9IZZWSfpnKekr6v1Q2U1KPNAv415zzrpV0RtqulvSr1M50ST1T+fGSpkmaldrrIqk7cBbw3fQ7+7SkyyRdkM7Z3Pv0itT+C4XefwXGLXcMvgz0BC4DiIgAHgfqG38zM7NtTkMSxQOA30XEgcA7ZP7HCfAH4AcRcRAwF/hxPefnGgSMjYhhwJeAh9PMzABgdj3nDE19HgScJKlc0ieAU4DD0vm1wGmpfltgWkQMiIh/5LX1JJnZM4C9gL5p+9PAE/W1q8zS8iXA8IgYBFSQmWnKakcmwbojIm4qcA0/Bh5LYzgJyJ2xOwC4Lh3bAFwBfBYYCAyRNKaeccnVPY3T54HrJbXOPShpP2B5RLwHEBF1wLeAu4EFEfFETvUK3h+j+pyY4hsADAeuTIn/scBo4JCIGAD8KtX/E5n30AAyM3+vN+CaVkREf+Ba4Dep7B/AJyPiYGAi8P2IqAauB36dZvqezGtnc+/TkogYCpxHgfdv/rjllHcnMyt7WkTU5BxqyNiZmZltMxqyBPxyRGSTuBlAd0kdgF0iYmoqvw24qwFtPRIRb6ftZ4FbJLUC7svpo9A5/waQdA9wOFADDAaelQRQCryZ6teSSYAKeRI4T1JfYB6wq6SuZGYezwW+Wk+7nySTVD6VyncCns5p9y/AryLiT/X0ezhwAkBEPCRpec6xxRHxTNoeAjweEcvS9f4JOAK4r552s+5Myd9CSS8Bfdg08e4KLMs9ISJmK3P/6XV5bb0J7Fmkv8PJJMW1wBuSpqbYhwG3RsSa1MfbktoDe0XEvalsXbq2Il1wR87PX6ftvYE/p9/ZTsDLm2ugAe/Te9LPGWSS7XwfGLc0w/xH4NKIWJRXv96xk/RN4JsAJZ123VzYZmZmW42GzCjmzqbUUjy5rMlpt3XesXezG2kW6whgCTBe0umSTtD7DwqUZ6vmtRGAgNty7hfrHRGXpePrUgKDpENy2hsVEUuAXYBjgCfIJI4nA6sjYtVm2hWZhDVb3jcivp4T01PAMUrZj6Szc/otlnS9W+R47nVn5Y9roTHKtbbAOQB16ZWrdaq/peW+T7L95ooC2+OAa9NM438WOOfDyr6363tfFxq3S4DXI6LQh3rVO3YRcWNElEdEecuydoWqmJmZbXU+0sMTEbECWJ5zX9dXgOysTTWZWTmAsfW1IWlf4I20VHszMCgi7s1JxipS1c9J6qjMvZFjyCRljwJjJe2e2uqY2suPc1pOe5NT8TNklhqzieIF6SebafcZ4LCce+XaSuqV09WPgOVk7vcjIn6X0++/Uswnp3OPBuqbUpoODEv3L7YETs0Z1zckfUKZB15OyDvvJEktJPUA9gcW5B1/gcIzZoX0InO/JJKGpvsn8z0JnCKppaTOZBL+6cAjwJlKD3RI6pgS8NeyS+iSdk7HFwN90/4uwFF5fZyS8zM7e9uBzB8WkJn9zVoFtM8Pssj7tCE2GTdJnwTOIM0MFrBx7MzMzLYHH+cp26+SuTdtDpn71bIfDXIV8C1Js4DNfWzMZ4DKVO8U4Jp66k0ns5Q8B7g7IioiYh6ZmZ0pqf9HyCwTNsSTZO5NWwTMBDqmMuprNy0FnwHckcqfJrO8m+u/gFJJv+KDfgIcnZZ6TwKWkkluNhERrwMXAX8HKoEZEfGXdPgi4K/AP/ngPX6vkBmnB4Gzssu7Oe2+C7yYTXSLOJLME7yQuZey0AzZvWR+H5XAY2TuFVwaEQ8Bk4EKSbPJJOGQSdDOTWP3T2CPiHgVuJNMYnUnMCuvj11T/f8CvpvKLgPukjQDeCun7v1AdjY6/x7B+t6nRRUYt58AbYC/a9OPyemRjueOnZmZ2TZPmYc1t07pKdjyiDinuWP5OCTtDNRGRI2kQ4Hfp4dlGqPt8cBfI2JSkXonAIMj4pIicU4FDk+xXgncHhFzGiPWhlLmSfnyiHirWN0miKXouKV6XYAJEZE/M/oBrXt0i+5XfL+xQrSthL/Cz1/hZ1vO3bNvRdKMiCgvXtsak7/ruWl0A+5My8brgQ98hM6WFhH3StqtSLVuwEXZJ3kj4sItH9nWrYHjBpmxO39Lx2NmZtaUtupEMSLGA+ObOYyPLSIWAgdvobbP+BB1by5yfCGw8OPG9HFFRPfmjiFXsXFLdZ5tiljMzMya0la99Gy2PSovL4+KioriFc3MbCMvPTePZvkKPzMzMzPb+jlRNDMzM7OCnCiamZmZWUFOFM3MzMysICeKZmZmZlaQn3o2a2KSVvHBr1ncUXVi02/Z2ZF5LN7nsXifxyKjE9A2Ijo3dyA7mq36cxTNtlML/BEPGZIqPBYZHov3eSze57HISOPQvbnj2BF56dnMzMzMCnKiaGZmZmYFOVE0a3o3NncAWxGPxfs8Fu/zWLzPY5HhcWgmfpjFzMzMzAryjKKZmZmZFeRE0WwLkHSMpAWSFkm6qMDxnSX9OR2fJql700fZNBowFt+TNE/SHEmPStq3OeJsCsXGIqfeFySFpO32adeGjIWkk9N74zlJE5o6xqbSgH8j3ST9XdKs9O9kZHPE2RQk3SLpTUlV9RyXpN+msZojaVBTx7ijcaJo1sgktQR+BxwL9AVOldQ3r9rXgeUR0RP4NXBF00bZNBo4FrOA8og4CJgE/Kppo2waDRwLJLUH/guY1rQRNp2GjIWkA4CLgcMi4kDgvCYPtAk08H1xCXBnRBwMfBG4rmmjbFLjgWM2c/xY4ID0+ibw+yaIaYfmRNGs8Q0FFkXESxGxHpgIjM6rMxq4LW1PAo6SpCaMsakUHYuI+HtErEm7zwB7N3GMTaUh7wuAn5L5w2FdUwbXxBoyFt8AfhcRywEi4s0mjrGpNGQsAihL2x2AfzVhfE0qIp4A3t5MldHAHyLjGWAXSV2bJrodkxNFs8a3F/Bqzv5rqaxgnYioAVYAuzVJdE2rIWOR6+vAg1s0ouZTdCzSMto+EfFAUwbWDBryvugF9JL0lKRnJG1ulmlb1pCxuAz4sqTXgL8B32ma0LZKH/a/KfYx+ZtZzGyrIOnLQDkwrLljaQ6SWgBXA2c0cyhbixIyy4ufITPL/ISk/hHxTrNG1TxOBcZHxP9IOhS4XVK/iKhr7sBs++cZRbPGtwTYJ2d/71RWsI6kEjLLSf9ukuiaVkPGAknDgR8CoyLivSaKrakVG4v2QD/gcUnVwCeBydvpAy0NeV+8BkyOiA0R8TLwApnEcXvTkLH4OnAnQEQ8DbQm893HO6IG/TfFGo8TRbPG9yxwgKT9JO1E5ubzyXl1JgNfTdtjgcdi+/xQ06JjIelg4AYySeL2eh8aFBmLiFgREZ0ionv6TttnyIxJRfOEu0U15N/IfWRmE5HUicxS9EtNGWQTachYvAIcBSDpE2QSxWVNGuXWYzJwenr6+ZPAioh4vbmD2p556dmskUVEjaRzgIeBlsAtEfGcpMuBioiYDPwvmeWjRWRu3P5i80W85TRwLK4E2gF3ped5XomIUc0W9BbSwLHYITRwLB4GjpY0D6gFLoyI7W7WvYFjcT5wk6Tvknmw5Yzt9A9LJN1B5g+ETumezB8DrQAi4noy92iOBBYBa4AzmyfSHYe/mcXMzMzMCvLSs5mZmZkV5ETRzMzMzApyomhmZmZmBTlRNDMzM7OCnCiamZmZWUFOFM3MzMysICeKZmZmZlaQE0UzMzMzK+j/A7CXJhkauzH/AAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "plot([\n", + " decomp_S_EG,\n", + " decomp_S_ER,\n", + " decomp_S_RO,\n", + " decomp_S_AG,\n", + " decomp_S_HO,\n", + "][::-1])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "[note: interpret the results]" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "broja", + "language": "python", + "name": "broja" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.4" + }, + "toc": { + "nav_menu": {}, + "number_sections": true, + "sideBar": true, + "skip_h1_title": false, + "toc_cell": false, + "toc_position": { + "height": "673px", + "left": "0px", + "right": "1228px", + "top": "110px", + "width": "212px" + }, + "toc_section_display": "block", + "toc_window_display": true + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} From 5fa44d9da54ea2c44ff2f9b0fb61a51c5d2b8847 Mon Sep 17 00:00:00 2001 From: heytitle Date: Thu, 26 Sep 2019 21:23:32 +0200 Subject: [PATCH 2/7] update notebook --- python/notebooks/us-1994-census.ipynb | 88 ++++++++++----------------- 1 file changed, 33 insertions(+), 55 deletions(-) diff --git a/python/notebooks/us-1994-census.ipynb b/python/notebooks/us-1994-census.ipynb index aa4cea7..6d4accb 100644 --- a/python/notebooks/us-1994-census.ipynb +++ b/python/notebooks/us-1994-census.ipynb @@ -15,44 +15,16 @@ "import math" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## General Structure (will be removed later)\n", - "\n", - "describe about the general structuure of this notebook. Ideally, it should contain\n", - "\n", - "- Problem definition\n", - "- Setting\n", - " - Data Preparation\n", - " - some statistics of us-census 1994\n", - " - auxiliary functions (helper) functions\n", - "\n", - "- Information theory\n", - " - basic quantities\n", - " - Entropy, KL-Divergence, Mutual Information, and some properites (chain rule, ... )\n", - " - information decomposision (BROJA & Pradeep's paper)\n", - " - MI = SI + CI + 2 UI\n", - " - breifly describe how the UI solver works. (simplex...), intuitively\n", - " - detail in appendix?" - ] - }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Introduction\n", "\n", - "Information and its theory are an important quantity that govern many fields, rangning from communication to machine learning. \n", - "\n", - "[figure src-> one receiver]\n", + "Information and its theory are an important quantity that govern many fields, rangning from communication to machine learning. In this tutorial, we are going to demonstate a novey approach of decomposing information into parts. Each part describes a certain aspect of the relationship of the variables.\n", "\n", - "In this tutorial, we are going to demonstate a way of decomposing information into parts. Each part describes a certain aspect of the relationship of the variables.\n", "\n", - "[figure src-> two receiver]\n", - "\n", - "We start the turotial with dataset description and some preprocessing steps ([Section X](#Data-Preparation)). Then, we define some auxiliary functions for visualizing results. Then, we talk about the foundation of information theory (Section X) and Information Decomposition. Along the way, we have code that implement or compute quantities of current interest." + "We start the turotial with dataset description and some preprocessing steps ([Section 2](#Data-Preparation)). Then, we define some auxiliary functions for visualizing results. Then, we talk about the foundation of information theory ([Section 3](#Basic-Information-Theory)) and Information Decomposition ([Section 4](#Information-Decomposition)). Thorough this tutorial, we provide code that implement or compute quantities of current interests." ] }, { @@ -422,7 +394,7 @@ "
Class:Distribution
Alphabet:('Female', 'Male') for all rvs
Base:linear
Outcome Class:tuple
Outcome Lenght:1
Gp(x)
Female0.33079450876815825
Male0.6692054912318418
" ], "text/plain": [ - "" + "" ] }, "execution_count": 10, @@ -495,7 +467,7 @@ "\n", "[note: revise this paragraph, make it relevant to what we're trying to do]\n", "\n", - "Information theory is foundation of many fields and technologies. The theory provides rigourous methods that enable us to develop ways of communication between source and receivers via **noisy** channels with least amount of error. \n", + "Information theory is a foundation of many fields and technologies. The theory provides rigourous methods that enable us to develop reliable ways of communication between senders and receivers via **noisy** channels. It is also a lens that helps us analyzing relationships between variables in pricipled ways. \n", "\n", "Some important quantities in Information Theory are:\n", "\n", @@ -669,25 +641,30 @@ "$$\n", "\\begin{align*}\n", "I(X; Y) &= D \\big(p(x, y) \\| p(x)p(y) \\big) \\\\\n", - "&= \\sum_{x \\mathcal{X}}\\sum_{y \\in \\mathcal{Y}} p(x, y) \\log \\frac{p(x, y)}{p(x)p(y)}\n", + "&= \\sum_{x \\in \\mathcal{X}}\\sum_{y \\in \\mathcal{Y}} p(x, y) \\log \\frac{p(x, y)}{p(x)p(y)}\n", "\\end{align*}\n", "$$\n", "\n", - "[note: Remove this part below]\n", - "If we continute the derivation, we have \n", - "\n", - "$$\n", - "I(X; Y) = H(X) - H(X|Y)\n", + "If we derive it further, we have" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "$$\n", - "\n", - "Intuitively, $I(X; Y)$ describes the overvall uncertainty of $X$ when $Y$ is known to us or vice versa. Please see [note: Appendix X][continue] for the derivation.\n", - "\n", - "[continue]: ..." + "\\begin{align*}\n", + "I(X; Y) &= \\sum_{x \\mathcal{X}}\\sum_{y \\in \\mathcal{Y}} p(x, y) \\log \\frac{p(x, y)}{p(x)p(y)} \\\\\n", + "&= \\sum_{x \\in \\mathcal{X}}\\sum_{y \\in \\mathcal{Y}} p(x, y) \\big( \\log p(x, y) - \\log p(x) - \\log p(y) \\big) \\\\\n", + "&= - H(X, Y) - \\sum_{x \\in \\mathcal{X}}\\sum_{y \\in \\mathcal{Y}} p(x, y) \\log p(x) - \\sum_{x \\mathcal{X}}\\sum_{y \\in \\mathcal{Y}} p(x, y) \\log p(y) \\\\\n", + "&= H(X) + H(Y) - H(X, Y)\n", + "\\end{align*}\n", + "$$" ] }, { "cell_type": "code", - "execution_count": 36, + "execution_count": 16, "metadata": {}, "outputs": [ { @@ -696,7 +673,7 @@ "0.037171387438320824" ] }, - "execution_count": 36, + "execution_count": 16, "metadata": {}, "output_type": "execute_result" } @@ -787,7 +764,7 @@ "source": [ "## Co-Information\n", "\n", - "**Definition:** Previouly known as **interaction information** [note: cite], co-Information $CoI(X;Y;Z)$ is defined by\n", + "**Definition:** Previouly known as **interaction information** [(McGill W. (1994)][mcgill], co-Information $CoI(X;Y;Z)$ is defined by\n", "\n", "$$\n", "CoI(X;Y;Z) = I(X;Y) - I(X;Y|Z).\n", @@ -797,7 +774,9 @@ "\n", "$$\n", "CoI(X;Y;Z) = SI(X; Y, Z) - CI(X; Y, Z).\n", - "$$\n" + "$$\n", + "\n", + "[mcgill]: https://ieeexplore.ieee.org/abstract/document/1057469\n" ] }, { @@ -860,9 +839,7 @@ "\n", "$$\n", "\\Delta_p = \\{ Q \\in \\mathbb{P}_{ \\mathcal{X} \\times \\mathcal{Y} \\times \\mathcal{Z} }: Q(X, Y) = P(X, Y) \\text{ and } Q(X, Z) = P(X, Z) \\}\n", - "$$\n", - "\n", - "[todo: add figure to illustrate this optimization procedure, maybe in appendix]" + "$$" ] }, { @@ -890,7 +867,7 @@ "
Class:Distribution
Alphabet:(('<=50K', '>50K'), ('Female', 'Male'), ('Amer-Indian-Eskimo', 'Asian-Pac-Islander', 'Black', 'Other', 'White'))
Base:linear
Outcome Class:tuple
Outcome Lenght:3
SGRp(x)
<=50KFemaleAmer-Indian-Eskimo0.008247547005324936
<=50KFemaleAsian-Pac-Islander0.005815377888949708
<=50KFemaleBlack0.07930424583381772
<=50KFemaleOther0.007555050560623192
<=50KFemaleWhite0.1936633251347222
<=50KMaleAmer-Indian-Eskimo0.0001981395952810085
<=50KMaleAsian-Pac-Islander0.01761756333138691
<=50KMaleBlack0.00475336950884044
<=50KMaleWhite0.44203582369502953
>50KFemaleAmer-Indian-Eskimo0.001018361885656181
>50KFemaleAsian-Pac-Islander0.0007180555469803716
>50KFemaleBlack0.009792116811899728
>50KFemaleOther0.0007677896847523185
>50KFemaleWhite0.023912637763622506
>50KMaleAmer-Indian-Eskimo8.725526069439487e-05
>50KMaleAsian-Pac-Islander0.007758342599998848
>50KMaleBlack0.0020932675154360938
>50KMaleWhite0.19466173037698395
" ], "text/plain": [ - "" + "" ] }, "execution_count": 21, @@ -1136,7 +1113,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "[note: writing some summary here]" + "From the figure above, we can see that `sex(Y)` contains substantial amount of information about `income(S)`, while `race(Z)` does not. Futhermore, the two attributes `sex(Y)` and `race(Z)` also share some information about `income(S)` but not in a complementary manner." ] }, { @@ -1150,7 +1127,6 @@ "- education and sex\n", "- education and race\n", "- race and occupation\n", - "- **education and occupation [note: didn't converged]**\n", "- age-group and sex\n", "- hours-per-week-group and occupation.\n", "\n", @@ -1249,13 +1225,13 @@ "metadata": {}, "outputs": [], "source": [ - "# didn't converge, can we remove it?\n", + "# didn't converge, should we remove it?\n", "# decomp_S_EO = compute_decomposition_from(dist_census, 'S', ['E', 'O'])" ] }, { "cell_type": "code", - "execution_count": 37, + "execution_count": 35, "metadata": {}, "outputs": [ { @@ -1285,7 +1261,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "[note: interpret the results]" + "When looking at `education` and `sex`, we can see that `education` uniquely contains considerable amount of information about `income` with respect to `sex`. The proportion is also large than what they convey about `income` in both shared and complementary aspects. Notably, `education` have large unique information about `income` when considering with `race`, and `occupation` also has a similar relative portion with respect to `race`.\n", + "\n", + "This way of decomposition provides us a new aspect for analyzing the interactions between variables (predictors and responses) at a granular level." ] } ], From 034b5b532e54bb10340b63e6c4e411ee40ff36d7 Mon Sep 17 00:00:00 2001 From: heytitle Date: Fri, 31 Jan 2020 13:10:51 +0100 Subject: [PATCH 3/7] add author details --- python/notebooks/us-1994-census.ipynb | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/python/notebooks/us-1994-census.ipynb b/python/notebooks/us-1994-census.ipynb index 6d4accb..3e8870a 100644 --- a/python/notebooks/us-1994-census.ipynb +++ b/python/notebooks/us-1994-census.ipynb @@ -24,7 +24,9 @@ "Information and its theory are an important quantity that govern many fields, rangning from communication to machine learning. In this tutorial, we are going to demonstate a novey approach of decomposing information into parts. Each part describes a certain aspect of the relationship of the variables.\n", "\n", "\n", - "We start the turotial with dataset description and some preprocessing steps ([Section 2](#Data-Preparation)). Then, we define some auxiliary functions for visualizing results. Then, we talk about the foundation of information theory ([Section 3](#Basic-Information-Theory)) and Information Decomposition ([Section 4](#Information-Decomposition)). Thorough this tutorial, we provide code that implement or compute quantities of current interests." + "We start the turotial with dataset description and some preprocessing steps ([Section 2](#Data-Preparation)). Then, we define some auxiliary functions for visualizing results. Then, we talk about the foundation of information theory ([Section 3](#Basic-Information-Theory)) and Information Decomposition ([Section 4](#Information-Decomposition)). Thorough this tutorial, we provide code that implement or compute quantities of current interests.\n", + "\n", + "This article is written by [Pattarawat Chormai](https://pat.chormai.org) and licensed under [CC-BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/)." ] }, { From 7b7cbe1191b5cc34a61acffe34530d4ef0bd2c5c Mon Sep 17 00:00:00 2001 From: Johannes Rauh Date: Fri, 31 Jan 2020 22:33:29 +0100 Subject: [PATCH 4/7] Fix some typos --- python/notebooks/us-1994-census.ipynb | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/python/notebooks/us-1994-census.ipynb b/python/notebooks/us-1994-census.ipynb index 3e8870a..175b649 100644 --- a/python/notebooks/us-1994-census.ipynb +++ b/python/notebooks/us-1994-census.ipynb @@ -21,8 +21,7 @@ "source": [ "# Introduction\n", "\n", - "Information and its theory are an important quantity that govern many fields, rangning from communication to machine learning. In this tutorial, we are going to demonstate a novey approach of decomposing information into parts. Each part describes a certain aspect of the relationship of the variables.\n", - "\n", + "Information and its theory are an important quantity that govern many fields, ranging from communication to machine learning. In this tutorial, we are going to demonstate a novel approach of decomposing information into parts. Each part describes a certain aspect of the relationship of the variables.\n", "\n", "We start the turotial with dataset description and some preprocessing steps ([Section 2](#Data-Preparation)). Then, we define some auxiliary functions for visualizing results. Then, we talk about the foundation of information theory ([Section 3](#Basic-Information-Theory)) and Information Decomposition ([Section 4](#Information-Decomposition)). Thorough this tutorial, we provide code that implement or compute quantities of current interests.\n", "\n", @@ -35,7 +34,7 @@ "source": [ "# Data Preparation\n", "\n", - "In this tutorial, we use **[US Census 1994][us-census]**, a publily available dataset, to demonstate the content of this turorial. The dataset contains individual's attributes, such as race, age, gender, and the level of income ( <= X, > X ). Hence, ML learners use it to train a classifier for predicting the level of income based on other attributes.\n", + "In this tutorial, we use **[US Census 1994][us-census]**, a publicly available dataset, to demonstate the content of this turorial. The dataset contains individual's attributes, such as race, age, gender, and the level of income ( $\\le$50 k, > 50 k). Hence, ML learners use it to train a classifier for predicting the level of income based on other attributes.\n", "\n", "\n", "Here, we are interested in explaing the relationship between these attributes and the income variable; therefore, we only use the training set.\n", @@ -285,9 +284,8 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "As you can see from the above, some attributes (columns) contains a space in the begining. Although these spaces do not\n", - "affect our computation, it is still good to clean it up. \n", - "\n", + "As you can see from the above, some attributes (columns) contains a space in the beginning. Although these spaces do not\n", + "affect our computation, it is still good to clean it up.\n", "This can been done by finding string columns (stored as `object`) and use Python's `strip` function to remove these prefix spaces." ] }, @@ -766,13 +764,13 @@ "source": [ "## Co-Information\n", "\n", - "**Definition:** Previouly known as **interaction information** [(McGill W. (1994)][mcgill], co-Information $CoI(X;Y;Z)$ is defined by\n", + "**Definition:** Also known as **interaction information** [(McGill W. (1994)][mcgill], co-information $CoI(X;Y;Z)$ is defined by\n", "\n", "$$\n", "CoI(X;Y;Z) = I(X;Y) - I(X;Y|Z).\n", "$$\n", "\n", - "With the chain rule of mutual information, we can write the identity above as\n", + "With the definition of information decomposition, we can write the identity above as\n", "\n", "$$\n", "CoI(X;Y;Z) = SI(X; Y, Z) - CI(X; Y, Z).\n", From 13abdbd47f1bfce47caad2cc7b2c41d7445dc94d Mon Sep 17 00:00:00 2001 From: heytitle Date: Sat, 8 Feb 2020 12:36:05 +0100 Subject: [PATCH 5/7] incorporate pradeep's fixes --- python/notebooks/us-1994-census.ipynb | 92 +++++++-------------------- 1 file changed, 24 insertions(+), 68 deletions(-) diff --git a/python/notebooks/us-1994-census.ipynb b/python/notebooks/us-1994-census.ipynb index 175b649..9c312d2 100644 --- a/python/notebooks/us-1994-census.ipynb +++ b/python/notebooks/us-1994-census.ipynb @@ -21,25 +21,19 @@ "source": [ "# Introduction\n", "\n", - "Information and its theory are an important quantity that govern many fields, ranging from communication to machine learning. In this tutorial, we are going to demonstate a novel approach of decomposing information into parts. Each part describes a certain aspect of the relationship of the variables.\n", + "This tutorial shows an application of the BROJA information decomposition on the **[US Census 1994][us-census]** income data set. The task is to relate a list of attributes or predictors with a binary target variable. The attributes include: sex, age, race, education level, occupation, hours-per-week, etc. The target is the yearly income, with values $>50$K and $\\leq50$K. \n", "\n", - "We start the turotial with dataset description and some preprocessing steps ([Section 2](#Data-Preparation)). Then, we define some auxiliary functions for visualizing results. Then, we talk about the foundation of information theory ([Section 3](#Basic-Information-Theory)) and Information Decomposition ([Section 4](#Information-Decomposition)). Thorough this tutorial, we provide code that implement or compute quantities of current interests.\n", "\n", - "This article is written by [Pattarawat Chormai](https://pat.chormai.org) and licensed under [CC-BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/)." + "This tutorial is written by [Pattarawat Chormai](https://pat.chormai.org) and licensed under [CC-BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/).\n", + "\n", + "[us-census]: https://archive.ics.uci.edu/ml/datasets/adult" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "# Data Preparation\n", - "\n", - "In this tutorial, we use **[US Census 1994][us-census]**, a publicly available dataset, to demonstate the content of this turorial. The dataset contains individual's attributes, such as race, age, gender, and the level of income ( $\\le$50 k, > 50 k). Hence, ML learners use it to train a classifier for predicting the level of income based on other attributes.\n", - "\n", - "\n", - "Here, we are interested in explaing the relationship between these attributes and the income variable; therefore, we only use the training set.\n", - "\n", - "[us-census]: https://archive.ics.uci.edu/ml/datasets/adult" + "# Data Preparation" ] }, { @@ -285,8 +279,7 @@ "metadata": {}, "source": [ "As you can see from the above, some attributes (columns) contains a space in the beginning. Although these spaces do not\n", - "affect our computation, it is still good to clean it up.\n", - "This can been done by finding string columns (stored as `object`) and use Python's `strip` function to remove these prefix spaces." + "affect our computation, it is still good to clean it up." ] }, { @@ -306,7 +299,7 @@ "source": [ "## Discretization\n", "\n", - "As you can see from the data exploration part, `age` and `hours-per-week` are continuous. This might be useful for some cases to treat them as they are. In this tutorial, we are interested in only groups of these values. Therefore, we need first need to perform discretization on this values. More precisely,\n", + "Some attributes such as age and hours-per-week are continuous. We discretize these attributes as follows: \n", "\n", "- We categorize `age` into four groups: [ '<24', '24-35', '36-50', '>50' ], and\n", "- We group `hours_per_week_group` into two groups: ['<=40', '>40']" @@ -329,11 +322,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Data Distribution\n", + "## Distribution\n", "\n", - "Once we have finished cleansing and discretizing the data, we are now ready to instantiate a [dit][dit]'s `Distribution` variable. This variable comes with necessary methods for dealing probabilistic operators, such as marginalization and conditioning.\n", - "\n", - "[dit]: https://github.com/dit/dit" + "The code expects a joint distribution over predictors and target to be passed in dit format." ] }, { @@ -465,22 +456,17 @@ "source": [ "# Basic Information Theory\n", "\n", - "[note: revise this paragraph, make it relevant to what we're trying to do]\n", - "\n", - "Information theory is a foundation of many fields and technologies. The theory provides rigourous methods that enable us to develop reliable ways of communication between senders and receivers via **noisy** channels. It is also a lens that helps us analyzing relationships between variables in pricipled ways. \n", + "We review some basic definitions from information theory. \n", "\n", - "Some important quantities in Information Theory are:\n", "\n", "## Entropy\n", "Let $X$ be a discreate random variable. Random variable $X$ takes values from $\\mathcal{X}$ and has probability mass function $p(x) = P\\{X=x\\}, x \\in \\mathcal{X} $.\n", "\n", - "**Definition:** Entropy $H(X)$ of a discrete random variable $X$ is defined by\n", + "**Definition:** Entropy $H(X)$ of a discrete random variable $X$ is defined as\n", "\n", "$$\n", "H(X) = - \\sum_{x \\in \\mathcal{X} } p(x) \\log_{2} p(x).\n", - "$$\n", - "\n", - "Because of $\\log_2$ in the equation, it is measured in terms of *bits*, and we omit writing the base of the log from now onwards." + "$$" ] }, { @@ -513,10 +499,10 @@ "\n", "Let $(X, Y)$ be a pair of discrete random variable with a joint distribution $p(x,y)$, $x \\in \\mathcal{X}$ and $y \\in \\mathcal{Y}$.\n", "\n", - "**Definition:** Joint Entropy $H(X,Y)$ is definied by\n", + "**Definition:** Joint Entropy $H(X,Y)$ is definied as\n", "\n", "$$\n", - "H(X, Y) = - \\sum_{x \\in \\mathcal{X}} \\sum_{y \\in \\mathcal{Y}} p(x, y) \\log p(x, y)\n", + "H(X, Y) = - \\sum_{x \\in \\mathcal{X}} \\sum_{y \\in \\mathcal{Y}} p(x, y) \\log p(x, y).\n", "$$" ] }, @@ -546,10 +532,10 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "**Definition:** Conditional Entropy $H(Y|X)$ is defined by \n", + "**Definition:** Conditional Entropy $H(Y|X)$ is defined as \n", "\n", "$$\n", - "H(Y|X) = - \\sum_{x \\in \\mathcal{X}} \\sum_{y \\in \\mathcal{Y}} p(x, y) \\log p(y|x)\n", + "H(Y|X) = - \\sum_{x \\in \\mathcal{X}} \\sum_{y \\in \\mathcal{Y}} p(x, y) \\log p(y|x).\n", "$$" ] }, @@ -574,23 +560,6 @@ "dit.shannon.conditional_entropy(dist_census, 'S', 'G') " ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "**Theorem:** Chain Rule of Joint Entropy $H(X, Y)$\n", - "\n", - "$$\n", - "H(X, Y) = H(X) + H(Y|X)\n", - "$$\n", - "\n", - "Proof can be found at [Cover, Thomas M and Thomas, Joy A's Elements of Information Theory, Theorem 2.2.1][element].\n", - "\n", - "Below, we verify the theorem computationally.\n", - "\n", - "[element]: https://www.wiley.com/en-it/Elements+of+Information+Theory,+2nd+Edition-p-9780471241959" - ] - }, { "cell_type": "code", "execution_count": 15, @@ -620,10 +589,10 @@ "\n", "Let $p$ and $q$ be two distributions with probability mass function $p(x)$ and $q(x)$ accordingly. Relative entropy is a distance between $p$ and $q$.\n", "\n", - "**Definition:** Relative entropy between two distributions $p$ and $q$ $D(p\\|q)$ is defined by \n", + "**Definition:** Relative entropy between two distributions $p$ and $q$ $D(p\\|q)$ is defined as \n", "\n", "$$\n", - "D(p\\|q) = \\sum_{x \\in \\mathcal{X}} p(x) \\log \\frac{p(x)}{q(x)}\n", + "D(p\\|q) = \\sum_{x \\in \\mathcal{X}} p(x) \\log \\frac{p(x)}{q(x)}.\n", "$$\n", "\n", "Another name of relative entropy is **Kullback-Leibler** divergence. Important properties of $D(p\\|q)$ are:\n", @@ -641,23 +610,10 @@ "$$\n", "\\begin{align*}\n", "I(X; Y) &= D \\big(p(x, y) \\| p(x)p(y) \\big) \\\\\n", - "&= \\sum_{x \\in \\mathcal{X}}\\sum_{y \\in \\mathcal{Y}} p(x, y) \\log \\frac{p(x, y)}{p(x)p(y)}\n", - "\\end{align*}\n", - "$$\n", - "\n", - "If we derive it further, we have" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "$$\n", - "\\begin{align*}\n", - "I(X; Y) &= \\sum_{x \\mathcal{X}}\\sum_{y \\in \\mathcal{Y}} p(x, y) \\log \\frac{p(x, y)}{p(x)p(y)} \\\\\n", + "&= \\sum_{x \\in \\mathcal{X}}\\sum_{y \\in \\mathcal{Y}} p(x, y) \\log \\frac{p(x, y)}{p(x)p(y)} \\\\\n", "&= \\sum_{x \\in \\mathcal{X}}\\sum_{y \\in \\mathcal{Y}} p(x, y) \\big( \\log p(x, y) - \\log p(x) - \\log p(y) \\big) \\\\\n", "&= - H(X, Y) - \\sum_{x \\in \\mathcal{X}}\\sum_{y \\in \\mathcal{Y}} p(x, y) \\log p(x) - \\sum_{x \\mathcal{X}}\\sum_{y \\in \\mathcal{Y}} p(x, y) \\log p(y) \\\\\n", - "&= H(X) + H(Y) - H(X, Y)\n", + "&= H(X) + H(Y) - H(X, Y).\n", "\\end{align*}\n", "$$" ] @@ -688,7 +644,7 @@ "source": [ "### Contitional Mutual Information\n", "\n", - "**Definition:** For three discrete random variables $X, Y, Z$, the conditional mutual information $I(X; Y |Z)$ is defined by\n", + "**Definition:** For three discrete random variables $X, Y, Z$, the conditional mutual information $I(X; Y |Z)$ is defined as\n", "\n", "$$\n", "I(X; Y |Z) = H(X|Z) - H(X|Y,Z).\n", @@ -1269,9 +1225,9 @@ ], "metadata": { "kernelspec": { - "display_name": "broja", + "display_name": "vdb-new", "language": "python", - "name": "broja" + "name": "vdb-new" }, "language_info": { "codemirror_mode": { @@ -1283,7 +1239,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.4" + "version": "3.7.6" }, "toc": { "nav_menu": {}, From 75509078cde8cb63518156cac7c86fcbd1c158ff Mon Sep 17 00:00:00 2001 From: heytitle Date: Sat, 8 Feb 2020 12:38:54 +0100 Subject: [PATCH 6/7] fix typo --- python/notebooks/us-1994-census.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/notebooks/us-1994-census.ipynb b/python/notebooks/us-1994-census.ipynb index 9c312d2..a16ebcf 100644 --- a/python/notebooks/us-1994-census.ipynb +++ b/python/notebooks/us-1994-census.ipynb @@ -720,7 +720,7 @@ "source": [ "## Co-Information\n", "\n", - "**Definition:** Also known as **interaction information** [(McGill W. (1994)][mcgill], co-information $CoI(X;Y;Z)$ is defined by\n", + "**Definition:** Also known as **interaction information** [(McGill W. (1994)][mcgill], co-information $CoI(X;Y;Z)$ is defined as\n", "\n", "$$\n", "CoI(X;Y;Z) = I(X;Y) - I(X;Y|Z).\n", From 319551def697cbd772130d3f7b1c47a35911305d Mon Sep 17 00:00:00 2001 From: heytitle Date: Sat, 8 Feb 2020 22:36:14 +0100 Subject: [PATCH 7/7] incorporate fixes for other sections --- python/notebooks/us-1994-census.ipynb | 178 ++++++++++++-------------- 1 file changed, 83 insertions(+), 95 deletions(-) diff --git a/python/notebooks/us-1994-census.ipynb b/python/notebooks/us-1994-census.ipynb index a16ebcf..52adc05 100644 --- a/python/notebooks/us-1994-census.ipynb +++ b/python/notebooks/us-1994-census.ipynb @@ -125,7 +125,7 @@ " \n", " \n", " \n", - " 0\n", + " 0\n", " 39\n", " State-gov\n", " 77516\n", @@ -143,7 +143,7 @@ " <=50K\n", " \n", " \n", - " 1\n", + " 1\n", " 50\n", " Self-emp-not-inc\n", " 83311\n", @@ -161,7 +161,7 @@ " <=50K\n", " \n", " \n", - " 2\n", + " 2\n", " 38\n", " Private\n", " 215646\n", @@ -179,7 +179,7 @@ " <=50K\n", " \n", " \n", - " 3\n", + " 3\n", " 53\n", " Private\n", " 234721\n", @@ -197,7 +197,7 @@ " <=50K\n", " \n", " \n", - " 4\n", + " 4\n", " 28\n", " Private\n", " 338409\n", @@ -346,7 +346,7 @@ "\n", "# aliases\n", "rvs_names = [\n", - " 'S', # income\n", + " 'X', # income\n", " 'E', # education\n", " 'G', # sex\n", " 'R', # race\n", @@ -385,7 +385,7 @@ "
Class:Distribution
Alphabet:('Female', 'Male') for all rvs
Base:linear
Outcome Class:tuple
Outcome Lenght:1
Gp(x)
Female0.33079450876815825
Male0.6692054912318418
" ], "text/plain": [ - "" + "" ] }, "execution_count": 10, @@ -416,7 +416,7 @@ "Base: linear\n", "Outcome Class: tuple\n", "Outcome Length: 1\n", - "RV Names: ('S',)\n", + "RV Names: ('X',)\n", "\n", "x p(x)\n", "('<=50K',) 0.8905394113824158\n", @@ -430,7 +430,7 @@ "Base: linear\n", "Outcome Class: tuple\n", "Outcome Length: 1\n", - "RV Names: ('S',)\n", + "RV Names: ('X',)\n", "\n", "x p(x)\n", "('<=50K',) 0.6942634235888022\n", @@ -440,7 +440,7 @@ ], "source": [ "# conditional probablity P(S|G).\n", - "marginal, cdists = dist_census.condition_on('G', rvs='S')\n", + "marginal, cdists = dist_census.condition_on('G', rvs='X')\n", "\n", "for i, (c, d) in enumerate(zip(cdists, marginal.zipped())):\n", " print(\"\")\n", @@ -523,9 +523,9 @@ } ], "source": [ - "# H(S, G)\n", - "p_SG = dist_census.marginal('SG')\n", - "dit.shannon.entropy(p_SG)" + "# H(X, G)\n", + "p_XG = dist_census.marginal('XG')\n", + "dit.shannon.entropy(p_XG)" ] }, { @@ -556,8 +556,8 @@ } ], "source": [ - "# H(S|G)\n", - "dit.shannon.conditional_entropy(dist_census, 'S', 'G') " + "# H(X|G)\n", + "dit.shannon.conditional_entropy(dist_census, 'X', 'G') " ] }, { @@ -577,8 +577,8 @@ } ], "source": [ - "dit.shannon.entropy(p_SG) \\\n", - " == dit.shannon.entropy(p_G) + dit.shannon.conditional_entropy(dist_census, 'S', 'G') " + "dit.shannon.entropy(p_XG) \\\n", + " == dit.shannon.entropy(p_G) + dit.shannon.conditional_entropy(dist_census, 'X', 'G') " ] }, { @@ -635,7 +635,7 @@ } ], "source": [ - "dit.shannon.mutual_information(dist_census, 'S', 'G')" + "dit.shannon.mutual_information(dist_census, 'X', 'G')" ] }, { @@ -668,9 +668,9 @@ } ], "source": [ - "# I(S, H | O)\n", - "dit.shannon.conditional_entropy(dist_census, 'S', 'O') - \\\n", - " dit.shannon.conditional_entropy(dist_census, 'S', 'HO')" + "# I(X, H | O)\n", + "dit.shannon.conditional_entropy(dist_census, 'X', 'O') - \\\n", + " dit.shannon.conditional_entropy(dist_census, 'X', 'HO')" ] }, { @@ -679,10 +679,11 @@ "source": [ "# Information Decomposition\n", "\n", - "Let consider the setting that we have three random variables $X, Y, Z$. We are interested in knowing about $X$, but it is not observable. We can only observe the values of $Y$ and $Z$. \n", + "Suppose that we have three jointly distributed random variables, $X, Y, Z$. Suppose that we are interested in knowing $X$, but we can only observe $Y$ and / or $Z$.\n", + "\n", + "\n", "![](https://i.imgur.com/GpHQ6MW.png)\n", "\n", - "Utimately, we would like to quantify how much we know about $X$ based on the information of $Y$ and $Z$. More precisely, $I(X; (Y, Z))$ is the total information of $X$ that $(Y, Z)$ contains.\n", "\n", "\n", "[Bertschinger et al. (2014)][paper] proposes one approach to decompose $I(X; (Y, Z))$ into four quantities:\n", @@ -699,7 +700,6 @@ "- $CI(X; Y, Z)$ is complimentary (synergic) information that $Y$ and $Z$ have about $X$ when considering them together,\n", "- $UI(X; Y \\backslash Z)$ is unique information that only $Y$ has about $X$ (in respect to $Z$), and vice versa. \n", "\n", - "With the formulation above, these four quantities have to be non-negative.\n", "\n", "## Shared Information\n", "Furthermore, we have the following equilities:\n", @@ -720,17 +720,16 @@ "source": [ "## Co-Information\n", "\n", - "**Definition:** Also known as **interaction information** [(McGill W. (1994)][mcgill], co-information $CoI(X;Y;Z)$ is defined as\n", + "**Definition:** The coinformation is defined as [(McGill W. (1994)][mcgill]\n", "\n", "$$\n", - "CoI(X;Y;Z) = I(X;Y) - I(X;Y|Z).\n", + "\\begin{align*}\n", + "CoI(X;Y;Z) &= I(X;Y) - I(X;Y|Z) \\\\\n", + "&= SI(X; Y, Z) - CI(X; Y, Z)\n", + "\\end{align*}\n", + ".\n", "$$\n", "\n", - "With the definition of information decomposition, we can write the identity above as\n", - "\n", - "$$\n", - "CoI(X;Y;Z) = SI(X; Y, Z) - CI(X; Y, Z).\n", - "$$\n", "\n", "[mcgill]: https://ieeexplore.ieee.org/abstract/document/1057469\n" ] @@ -752,8 +751,8 @@ } ], "source": [ - "# compute CoI(S; E; G) using dit\n", - "dit.multivariate.coinformation(dist_census, 'SEG')" + "# compute CoI(X; E; G) using dit\n", + "dit.multivariate.coinformation(dist_census, 'XEG')" ] }, { @@ -773,8 +772,8 @@ } ], "source": [ - "# compute CoI(S; H; G) using dit\n", - "dit.multivariate.coinformation(dist_census, 'SHG')" + "# compute CoI(X; H; G) using dit\n", + "dit.multivariate.coinformation(dist_census, 'XHG')" ] }, { @@ -783,7 +782,8 @@ "source": [ "## Unique Information\n", "\n", - "From above, we have everything in place except $UI(\\cdot)$ that we haven't defined yet. Bertschinger et al. (2014) define the unique information as follows:\n", + "To compute the information decomposition, it suffices to specify either a measure for $SI$, for $CI$ or for $UI$.\n", + "Bertschinger et al. (2014) define the unique information as follows:\n", "\n", "$$\n", "UI(X; Y \\backslash Z) = \\min_{Q \\in \\Delta_p} I_Q(X; Y|Z), \n", @@ -804,12 +804,12 @@ "metadata": {}, "outputs": [], "source": [ - "# find q for S, G, R (income, race, gender)\n", - "q_SGR = admUI.computeQUI(distSXY = dist_census.marginal('SGR'))\n", + "# find q for X, G, R (income, race, gender)\n", + "q_XGR = admUI.computeQUI(distSXY = dist_census.marginal('XGR'))\n", "\n", "# due to the fact that computeQUI rename variables to SXY\n", - "# we need to rename them back to SRG\n", - "q_SGR.set_rv_names(\"SGR\")" + "# we need to rename them back to XGR\n", + "q_XGR.set_rv_names(\"XGR\")" ] }, { @@ -820,10 +820,10 @@ { "data": { "text/html": [ - "
Class:Distribution
Alphabet:(('<=50K', '>50K'), ('Female', 'Male'), ('Amer-Indian-Eskimo', 'Asian-Pac-Islander', 'Black', 'Other', 'White'))
Base:linear
Outcome Class:tuple
Outcome Lenght:3
SGRp(x)
<=50KFemaleAmer-Indian-Eskimo0.008247547005324936
<=50KFemaleAsian-Pac-Islander0.005815377888949708
<=50KFemaleBlack0.07930424583381772
<=50KFemaleOther0.007555050560623192
<=50KFemaleWhite0.1936633251347222
<=50KMaleAmer-Indian-Eskimo0.0001981395952810085
<=50KMaleAsian-Pac-Islander0.01761756333138691
<=50KMaleBlack0.00475336950884044
<=50KMaleWhite0.44203582369502953
>50KFemaleAmer-Indian-Eskimo0.001018361885656181
>50KFemaleAsian-Pac-Islander0.0007180555469803716
>50KFemaleBlack0.009792116811899728
>50KFemaleOther0.0007677896847523185
>50KFemaleWhite0.023912637763622506
>50KMaleAmer-Indian-Eskimo8.725526069439487e-05
>50KMaleAsian-Pac-Islander0.007758342599998848
>50KMaleBlack0.0020932675154360938
>50KMaleWhite0.19466173037698395
" + "
Class:Distribution
Alphabet:(('<=50K', '>50K'), ('Female', 'Male'), ('Amer-Indian-Eskimo', 'Asian-Pac-Islander', 'Black', 'Other', 'White'))
Base:linear
Outcome Class:tuple
Outcome Lenght:3
XGRp(x)
<=50KFemaleAmer-Indian-Eskimo0.008247547005324936
<=50KFemaleAsian-Pac-Islander0.005815377888949708
<=50KFemaleBlack0.07930424583381772
<=50KFemaleOther0.007555050560623192
<=50KFemaleWhite0.1936633251347222
<=50KMaleAmer-Indian-Eskimo0.0001981395952810085
<=50KMaleAsian-Pac-Islander0.01761756333138691
<=50KMaleBlack0.00475336950884044
<=50KMaleWhite0.44203582369502953
>50KFemaleAmer-Indian-Eskimo0.001018361885656181
>50KFemaleAsian-Pac-Islander0.0007180555469803716
>50KFemaleBlack0.009792116811899728
>50KFemaleOther0.0007677896847523185
>50KFemaleWhite0.023912637763622506
>50KMaleAmer-Indian-Eskimo8.725526069439487e-05
>50KMaleAsian-Pac-Islander0.007758342599998848
>50KMaleBlack0.0020932675154360938
>50KMaleWhite0.19466173037698395
" ], "text/plain": [ - "" + "" ] }, "execution_count": 21, @@ -832,7 +832,7 @@ } ], "source": [ - "q_SGR" + "q_XGR" ] }, { @@ -852,13 +852,13 @@ } ], "source": [ - "# H(S|G,R)\n", - "h_SgGR = dit.shannon.conditional_entropy(q_SGR, 'S', 'GR')\n", + "# H(X|G,R)\n", + "h_XgGR = dit.shannon.conditional_entropy(q_XGR, 'X', 'GR')\n", "\n", - "# UI(S; G \\ R) = I_Q { S; G \\ R }\n", - "ui_SG_R = dit.shannon.conditional_entropy(q_SGR, 'S', 'R') - h_SgGR\n", + "# UI(X; G \\ R) = I_Q { X; G \\ R }\n", + "ui_XG_R = dit.shannon.conditional_entropy(q_XGR, 'X', 'R') - h_XgGR\n", "\n", - "ui_SG_R" + "ui_XG_R" ] }, { @@ -878,10 +878,10 @@ } ], "source": [ - "# UI(S; R \\ G) = I_Q { S; R \\ G }\n", - "ui_SR_G = dit.shannon.conditional_entropy(q_SGR, 'S', 'G') - h_SgGR\n", + "# UI(X; R \\ G) = I_Q { X; R \\ G }\n", + "ui_XR_G = dit.shannon.conditional_entropy(q_XGR, 'X', 'G') - h_XgGR\n", "\n", - "ui_SR_G " + "ui_XR_G " ] }, { @@ -902,11 +902,11 @@ ], "source": [ "# compute shared information\n", - "si_SGR = dit.shannon.mutual_information(q_SGR, 'S', 'G') - ui_SG_R \n", - "si_SRG = dit.shannon.mutual_information(q_SGR, 'S', 'R') - ui_SR_G\n", + "si_XGR = dit.shannon.mutual_information(q_XGR, 'X', 'G') - ui_XG_R \n", + "si_XRG = dit.shannon.mutual_information(q_XGR, 'X', 'R') - ui_XR_G\n", "\n", "# sanity check: by the definition of shared information\n", - "si_SGR == si_SRG" + "si_XGR == si_XRG" ] }, { @@ -927,8 +927,8 @@ ], "source": [ "# by the definition of co-information\n", - "ci_SGR = si_SRG - dit.multivariate.coinformation(q_SGR, 'SGR')\n", - "ci_SGR" + "ci_XGR = si_XRG - dit.multivariate.coinformation(q_XGR, 'XGR')\n", + "ci_XGR" ] }, { @@ -949,14 +949,14 @@ ], "source": [ "# sanity check: by the definition of Bertschinger et al. (2014)'s information decomposition\n", - "dit.shannon.mutual_information(q_SGR, 'S', 'GR') == ui_SG_R + ui_SR_G + si_SRG + ci_SGR " + "dit.shannon.mutual_information(q_XGR, 'X', 'GR') == ui_XG_R + ui_XR_G + si_XRG + ci_XGR " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "# Result Visualization" + "# The information decomposition for the US census dataset" ] }, { @@ -972,8 +972,8 @@ "latex_labels = [\n", " '$SI$',\n", " '$CI$',\n", - " '$UI(S; Y \\\\backslash Z)$',\n", - " '$UI(S; Z \\\\backslash Y)$'\n", + " '$UI(X; Y \\\\backslash Z)$',\n", + " '$UI(X; Z \\\\backslash Y)$'\n", "]\n", "\n", "# define colour for ploting\n", @@ -992,7 +992,7 @@ "outputs": [ { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgwAAABXCAYAAABoUB0SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAXFUlEQVR4nO3de1SU550H8O8PBmW4qgFv8YIFgRAQRCBl6wnxlAASBTXERmzTeFJ3jWatG+Oe5KSeY7Pt7rYx2RiaC+omtu4i9bIxWk2D2zS0JgQC4sCESBwjKCgGI3IJTGFmnv2DwR1wYMZwGcHv5xyOM8/7zDu/3wwyv3me930fUUqBiIiIaCBurg6AiIiIbn8sGIiIiMghFgxERETkEAsGIiIicogFAxERETnEgoGIiIgc0rg6ACIiosEoKyubrNFodgOIBL8If1sWAHqTyfSTBQsWfGWvAwsGIiIa1TQaze6pU6feExgY2OTm5saLC30LFotFGhsbIxoaGnYDyLDXh5UYERGNdpGBgYEtLBa+PTc3NxUYGNiM7lEa+31GMB4iIqLh4MZiYfCsr2G/dQELBiIiInKIBQMRERE5xIKBiIhoCNTW1nosWbLkO5GRkfcEBQVF3nfffaEA8OKLLwZkZ2fPdnV8g8WzJIiIaEwJP/ibBUO5vzNZT5U502/VqlVz1qxZ07h27domACgpKdECQGVlpVdUVFT7UMbkChxhICIiGiSTyYSSkhLf1NTU1p62hISEDgCoqqrSxsTEdLguuqHBgoGIiGiQNBoNEhMTW+bPn39vdnb27IKCAu+ebWfPntXGx8ezYCAiIiKgsLDwbF5e3jk/Pz/z8uXLQ/fu3TvBYDB4aLVaS0BAgNnV8Q0Wj2EgIiIaAm5ubkhNTW1LTU1tu379urtOp9OOGzdOhYeHj/rjFwCOMBAREQ3aoUOH/IxGowBAfX29pqSkxCctLa1Fp9NpIyIiRv10BMCCgYiIaNAOHDgwce7cufeGhYVFpKWlzd26deul5OTkb/R6vXbevHljomAQpXg1TSIiGr10Ol1NdHT0VVfHMRbodLqA6OjoIHvbOMJAREREDrFgICIiIodYMBAREZFDLBiIiIjIIRYMRERE5BALBiIiInKIBQMRERE5xIKBiIiIHOJaEi4SEBCggoKCXB0GEdGoUlZWdlUpFejqOPrz1ltvTXzllVemdHZ2uplMJsnMzLw2derUrvLycu+8vLxaV8c3GCwYXKTN3wttzz7u6jBuWwf/Nc/VIRDRMIg89fGgHi8ijj90H52yYFBP0lf+lTJnuuXk5NyVm5s7+d133zUEBwd3tba2ur366qsBlZWVXlFRUaN+ASpOSRAREQ3StWvX3LZu3Tpz//7954KDg7sAwNfX1/L8889/VVVVpY2JiRn160mwYCAiIhqkvLy8idHR0W0RERGdfbedPXtWGx8fz4KBiIjoTqfX67VRUVE3FQUGg8FDq9VaAgICzK6IayixYCAiIhokb29vs8Viuam9rKzMKzw8fNQfvwCwYCAiIhq0JUuWNB89enTixYsXNQDQ0dEhL730UoBOp9NGRESM+ukIgGdJEBERDdqiRYvan3322UsPPvhgqMVigclkkkceeeTrzz//XJuent7s6viGAgsGIiIaW5w8DXKobdiw4dqGDRuuueK5RwKnJIiIiMghlxQMIvKKiNwvIr8UkV/ZtM8WkS9FZIKI5IvIXFfEZ49NzDNF5HSfn5aePETkf0VkoqvjJSIiGkojXjCIyF0AvquU+guAXwBYJiL3WDfvALBVKXUdwBsA/vkW9z0sUyy2MSulLiqlYnp+APwIQDOA/7B23wtg/XDEQURE5CoDFgwi4i0ix0REJyJ6EfmBtX2BiBSKSJmIvC8i00REIyKfisgD1j7/JiK/tLPbhwH8EQCUUh0A/gnAayKSDsBXKfXf1n5/BZDsqAgQkQ+t3/5LAfxURJaKSLGIlFu/7U+x9vMRkbdFpFJEKkTkYWt7iogUicgpETkgIj4DxdznuT0B5AHYoJRqsDYfAbBqoJiJiIhGG0cjDGkALimlopVSkQD+KCIeAHIAZCmlFgB4C8AvlVImAI8DeENEkq2P/bmdfX4PwI0DUpRSxwE0AfgtbL6ZK6UsAAwAop3IY5xSKk4p9RKAk+geDZgPIB//P0qxFUCzUipKKTUPwAciEgDgZwCSlVKxAEoBPO0oZhu/BnBSKXXEJu4mAOOtoxK9iMjfi0ipiJSaW9qcSIuIiOj24GgIvxLAS9b5+T8opf4qIpEAIgGcEBEAcAdwGQCUUp+JyF4AfwCQqJS66RKZAKYBaOzT9hoArVKquk/7VwCmw/6Hta3f29yeAeD3IjINwDgA563tyQAe7emklGoSkSUAIgB8ZM1lHIAiZ2IWkcXWfdpb5KQn7q9tG5VSOwHsBADP4FnKQU5ERES3jQELBqXUFyISCyAdwC9E5E8A3gHwmVIqsZ+HRQG4DmByP9s7AHj2abNYf/rytPZ35Bub2zkAXlZKHbFOj2wb4HEC4IRSytEUQq+YRWQygFwAmdZplW8bNxER0ajg6BiG6QDalVL/BeBFALEAqgEEikiitY+HiNxrvb0CwCQA9wPIEZEJdnb7OYAQJ+MLBaC37vt3IpLgxGP8AdRbb//Ypv0EgA02uU0E8AmA74lIiLXNW0RCnYj5LQA5Sqnyvh2le6hiKoAaJ2IlIiIaFRxNSUQBeFFELAC6ADyplOoUkSwAr4qIv3Ufr4jIFQD/DuD7SqmLIvIbdJ/18OM++zwG4B8A7B7oia0HK3bYHEw4D8AlJ3LaBuCAiDQB+ADAHGv7L9B9cKUegBnAz5VS/yMijwPYJyLjrf1+BuCL/mK2FkoPAZgpIqtt+pxQSm1B9xTFJ9ZjOoiIaITpY//O3lTxtxZ56mOnLgS1bt26GZMnT+564YUXrgCAxWKBv79/TH19fUVubu6k8vJy77y8vNq2tjZZtGhRaFFRUbVGo0Ftba3Hhg0bZtbU1Ixva2tznzJlSmdxcXHfzyGcOnXKc+nSpXMvXLhQ6e7uDpPJhKSkpLm1tbWe586d048fP/7GVHdWVlaQXq/36rl/9epVj66uLrly5Ypu4cKFoUVFRdUeHh639Do4mpJ4H8D7dtpPo3sUoa9Qmz6v9rPPv1rPoJhgPX0SSqkPAXzYp2s2uof9ISJ+AM4qpers7O+BPvffBfCunX5tuLl4gVLqAwDx9mLtJ+YidE9l9OdHAF4faH9ERDT2VFVVaVNSUlp67p85c2bcpEmTTH5+fpbKykqvqKiodgDIyckJyMjIaNJouj+CV61aNWfNmjWNa9eubQKAkpISrb39x8bGGoODg435+fn+q1evbt64cePdISEhxuDg4L+dOHHCZ8mSJa09fQ8ePFhjG0dSUlL4jh07ajw9PVVSUlLL7t27Jz355JO3dFVKV13pcTOAWQ76XEf3mRNQSrUopR4Z9qgG5kzMAKBXSv1puIMhIqLbS3V1tTYuLu7GypTWlSo7gO5iIiYmpgMA9u/ff9fKlSuvA4DJZEJJSYlvamrqjQ/7hISEDgBISkoKqamp6TUMsGnTpiu5ubmT9+zZM6G4uNhn586ddenp6c3Hjx/3sxfT5cuXNWlpaaHPPPPMpdWrVzcDQFZW1vX8/PxJt5qfSwoGpVSxUqrCQZ+3b6dhfWditvbbNRLxEBHR7aOxsdHdZDLJrFmzbnxu6XQ6bWRkZDsAnD17VhsfH99hNBrl4sWL48PCwjoBQKPRIDExsWX+/Pn3Zmdnzy4oKPDueXxhYaEhKCioy/Z5VqxY0dLQ0OCxbdu2Ge+888658ePHq4ceeqj15MmTvn1jam1tdVu8eHFIZmbmtS1btlztaY+Pj++oqKjw7tvfEa4lQURENEilpaXasLCwXmfH6fV6bXR0dIfBYPDQarWWgIAAc0NDg8bX17fXl+HCwsKzeXl55/z8/MzLly8P3bt3r70TBm6Ii4v7Zv369Vd6ihNfX1+Ln5+f2XY0wmQyITMz8zshISHGHTt29Dr+T6PRwMPDQzU1Nd1SDcCCgYiIaJCqqqo8g4KC/tZz32w2o6yszCclJaXVOjXRDgDe3t6Wzs7OXp+9bm5uSE1NbXvzzTfrli5dek2n09k9hqFHdXW154IFC9pt25KTk1sOHz58Y1risccem20ymWTfvn019vbR1dUlXl5et3Q9IBYMREREgzRnzpzO06dPe7W3twsAPPfcc9MSEhLapk6datbpdNqIiIgOAAgMDDSbzWbp6Xfo0CE/o9EoAFBfX68pKSnxSUtLawGAxMTE0PPnz990KoPBYNDGxcX1Gs3IyMhoLigo8AeAzZs3T6uoqPA6evToOXtnQjQ0NLhPmDDBZHtWhTOGZbEmIiIiV3H2NMihtGLFipb33nvPb+7cuZEeHh4qNja2bc+ePbVA99REenp6c0/f+++/v7mgoMBn2bJlrQcOHJi4adOmWV5eXpZx48aprVu3XkpOTv7GbDajtrZ2fGBgYK/pC4PB4OHr62v29/fvdbHD2NhY45dffun52WefjX/55ZenT58+vfO+++4L79keHBxsPHbs2JcA8N577/klJyc34xaxYCAiIhoCu3btqgNw0+n/R44cOW97f+PGjY3bt2+fsmzZstb8/Pxae/s6deqUZ3p6epOPj0+vUYCQkJCuurq6SnuPiY2Nbauvr9copQYsmPbt2zdp+/btN8XpCAsGF4mcOBmlWU+5OozbF18bIhqjFi5c2F5aWtpiMpnQcy2GvuLj443x8fG39KGel5d3wVEfo9EoGRkZ1+fNm/c3R337EqW4BpIriEgrui+zfacKAHDVYa+xi/nfufnfybkDg89/tlIq0LZBp9PVREdH38mv6ZDR6XQB0dHRQfa2cYTBdaqVUnGuDsJVRKSU+TN/V8fhCndy7gDzH814lgQRERE5xIKBiIiIHGLB4Do7XR2AizH/O9udnP+dnDvA/EctFgwuopS6o//TMH/m7+oYXOVOzh1g/qMZCwYiIiJyiAXDMBORNBGpFhGDiDxrZ/t4Efm9dXuxiASNfJTDx4n8nxaRKhGpEJE/ichsV8Q5HBzlbtPvYRFRIjKmjhx3Jn8RWWl9/z8TkbyRjnE4OfG7P0tE/iwi5dbf/3RXxDkcROQtEflKRPT9bBcRedX62lSISOxIx0i3jqdVDiMRcQfwGoAH0X31r09F5IhSqsqm2xMAmpRSISLyKIBfAfjByEc79JzMvxxAnFKqXUSeBPBrjIH8ncwdIuIL4KcAikc+yuHjTP4iMhfAcwC+p5RqEpHJrol26Dn5/v8MwH6l1BsiEgHgOICgEQ92eOwB8BsAv+tn+2IAc60/9wF4w/rvkHg4Zs2CodoXABw6/bZTl5pet27djMmTJ3e98MILVwDAYrHA398/pr6+viI3N3dSeXm5d15eXm1bW5ssWrQotKioqFqj0aC2ttZjw4YNM2tqasa3tbW5T5kypbO4uPgLe8+RlZUVpNfrvXruX7161aOrq0vuvvvuv5WXl5/pWR/i1KlTnkuXLp174cKFSnd3d5hMJiQlJc1dtWrVtT179gQUFRVV21tnYiAcYRheCQAMSqkvlVKdAPIBZPbpkwngt9bbBwF8X0RkBGMcTg7zV0r9WSnVs+raJwBmjHCMw8WZ9x4A/gXdRaJxJIMbAc7kvxbAa0qpJgBQSn01wjEOJ2fyVwB6Vhf0B3AJY4RS6i8Arg3QJRPA71S3TwBMEJFpIxPd8KmqqtLGxMTcWBTqzJkz4yZNmmTy8/OzVFZWekVFRbUDQE5OTkBGRkZTz1UeV61aNSczM7NJr9d/XlNTo8/JybnY33McPHiw5syZM1VnzpypOnz4sMHd3V29/vrr5+Pi4r45ceKET0+/2NhYY3BwsDE/P98fADZu3Hh3SEiI8amnnvo6KSmpZffu3ZNuNT8WDMPrbgC2b3ydtc1uH6WUCUAzgLtGJLrh50z+tp4A8N6wRjRyHOZuHYadqZQ6NpKBjRBn3vtQAKEi8pGIfCIiaSMW3fBzJv9tAH4oInXoHl34x5EJ7bZwq38bRoXq6mptXFzcjWWnrctadwC9i4n9+/fftXLlyusAYDKZUFJS4puamtra87iEhIQOAEhKSgqpqamxOwxw+fJlTVpaWugzzzxzafXq1c3p6enNx48f97Pts2nTpiu5ubmT9+zZM6G4uNhn586ddQCQlZV1PT8/nwUDjU4i8kMAcQBedHUsI0FE3AC8DGCzq2NxIQ26h6QfALAKwC4RmeDSiEbWKgB7lFIzAKQD2Gv9vaBRqLGx0d1kMsmsWbNurC6p0+m0kZGR7QBw9uxZbXx8fIfRaJSLFy+ODwsL6wQAjUaDxMTElvnz59+bnZ09u6CgwLvn8YWFhYagoKCuvs/V2trqtnjx4pDMzMxrW7ZsuQoADz30UOvJkyd9bfutWLGipaGhwWPbtm0z3nnnnXM90xXx8fEdFRUV3n336wh/OYdXPYCZNvdnWNvs9hERDbqHJr8ekeiGnzP5Q0SSATwPIEMpdcsLotymHOXuCyASwIciUgPguwCOjKEDH5157+sAHFFKdSmlzgP4At0FxFjgTP5PANgPAEqpIgCe6F5n4U7g1N+G0aS0tFQbFhbWYdum1+u10dHRHQaDwUOr1VoCAgLMDQ0NGl9f315LVhcWFp7Ny8s75+fnZ16+fHno3r17+y2cTSYTMjMzvxMSEmLcsWPHjWksX19fi5+fn7nviERcXNw369evv2JbyGg0Gnh4eKimpqZbqgFYMAyvTwHMFZE5IjIOwKMAjvTpcwTAj623swB8oMbOimAO8xeR+QBy0V0sjKU57AFzV0o1K6UClFJBSqkgdB+/kaGUKnVNuEPOmd/9w+geXYCIBKB7iuLLkQxyGDmT/wUA3wcAEbkH3QVD44hG6TpHADxmPVviuwCalVKXXR3UYFRVVXkGBQXd+MJjNptRVlbmk5KS0mqdmmgHAG9vb0tnZ2evz143Nzekpqa2vfnmm3VLly69ptPptP09z2OPPTbbZDLJvn37avpuS05Objl8+HCvaYnq6mrPBQsWtPft29XVJV5eXrf0WcOCYRhZj0l4CsD7AD5H9xHRn4nICyKSYe32nwDuEhEDgKcB9Hv63WjjZP4vAvABcEBETotI3z+qo5KTuY9ZTub/PoCvRaQKwJ8BbFFKjYnRNSfz3wxgrYjoAOwD8PhY+bIgIvsAFAEIE5E6EXlCRNaJyDprl+PoLg4NAHYBWO+iUIfMnDlzOk+fPu3V3t4uAPDcc89NS0hIaJs6dapZp9NpIyIiOgAgMDDQbDabpaffoUOH/IxGowBAfX29pqSkxCctLa0FABITE0PPnz9/Y8Rg8+bN0yoqKryOHj16zt4ZDhkZGc0FBQX+tm0Gg0EbFxfXa+SjoaHBfcKECaaeKQpncXlrIiIa1W6X5a3Xrl074/jx4xM9PDxUbGxsW25u7sXAwEBzRkbGnPT09OZ169ZdA4CVK1fOzs7OvrZs2bLWRx99dPZHH33k6+XlZRk3bpx6+umnG9asWdNkNpsxc+bMqC+++ELv4+Ojqqurx4WHh0dNnz6909fX19zznMHBwcZjx47dGJkLDw+P0Ov1VRqNBgaDweOBBx4Ir6urq7SN8+2335748ccfe+/atauubw4DLW/NgoGIiEa126VgcNbJkye9tm/fPuXw4cPn++vz6aefeubm5gbs3r37pg/1gWRnZ896/PHHv05JSfmmvz4pKSnB27dvr5s3b95Nx4wNVDDwwk1EREQjaOHChe2lpaUtJpMJPddi6Cs+Pt4YHx9/S8UCAOTl5V0YaLvRaJSMjIzr9ooFRzjCQEREo9poG2G4nQ00wsCDHomIiMghFgxERETkEAsGIiIa7SwWi2WsrMHjMtbX0NLfdhYMREQ02ukbGxv9WTR8exaLRRobG/0B2F2SHOBZEkRENMqZTKafNDQ07G5oaIgEvwh/WxYAepPJ9JP+OvAsCSIiInKIlRgRERE5xIKBiIiIHGLBQERERA6xYCAiIiKHWDAQERGRQ/8HWgXGkkqxQH4AAAAASUVORK5CYII=\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgwAAABXCAYAAABoUB0SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAXIElEQVR4nO3dfVTUdb4H8PcHBmGA4cHA5wdcQAhBCMHW1hN5llBJoTVyU3dbO603H7psrXlPndbydnfvvZruzYfqoN5yzRDNboSrbnJrc3MlFcQBIglMRE0MFHkIEGbme/9gxos4MMPyMILv1zmcZr6/7/x+n89Azme+39/v9xWlFIiIiIi64uToAIiIiOjOx4KBiIiIbGLBQERERDaxYCAiIiKbWDAQERGRTSwYiIiIyCaNowMgIiLqiby8vGEajWY7gHDwi3BPmAAUGQyGX0+ZMuX7jhtZMBAR0YCm0Wi2jxgx4l5/f/8aJycn3lzoH2QymaSqqiqssrJyO4CkjttZiRER0UAX7u/vX8dioWecnJyUv79/LdpGam7f3s/xEBER9TYnFgu9w/w+Wq0NWDAQERGRTSwYiIiIyCYWDERERL3g/PnzLnPmzPlReHj4vQEBAeH333//RAB4/fXX/RYuXDje0fH1FK+SICKiQSV035Ypvbm/MynP5tnTb8GCBROeeuqpqiVLltQAwIkTJ7QAUFhY6B4REdHYmzE5AkcYiIiIeshgMODEiRO6mTNn1lvapk6d2gQAxcXF2qioqCbHRdc7WDAQERH1kEajwbRp0+ruu+++SQsXLhx/+PBhD8u20tJSbWxsLAsGIiIiAo4cOVKanp5+1svLy/izn/1s4nvvvedTVlbmotVqTX5+fkZHx9dTPIeBiIioFzg5OWHmzJkNM2fObLh+/bqzXq/XDhkyRIWGhg748xcAjjAQERH12IcffujV3NwsAHDp0iXNiRMnPGfNmlWn1+u1YWFhA346AmDBQERE1GMffPCBb3Bw8KSQkJCwWbNmBa9evfq7+Pj4H4qKirSTJ08eFAWDKMW7aRIR0cCl1+vLIyMjqx0dx2Ch1+v9IiMjAzq2c4SBiIiIbGLBQERERDaxYCAiIiKbWDAQERGRTSwYiIiIyCYWDERERGQTCwYiIiKyiQUDERER2cS1JBzEz89PBQQEODoMIqIBJS8vr1op5e/oODrzzjvv+L7xxhvDW1panAwGgyQnJ18bMWJEa35+vkd6evp5R8fXEywYHKTB2x0NLy52dBh3rH3/nu7oEIioD4SfOtaj14uI7Q/dJ4ZP6dFBOsq4kmdPt82bN9+TlpY27OOPPy4LDAxsra+vd9q0aZNfYWGhe0RExIBfgIpTEkRERD107do1p9WrV4/du3fv2cDAwFYA0Ol0ppdffvn74uJibVRU1IBfT4IFAxERUQ+lp6f7RkZGNoSFhbV03FZaWqqNjY1lwUBERHS3Kyoq0kZERNxWFJSVlblotVqTn5+f0RFx9SYWDERERD3k4eFhNJlMt7Xn5eW5h4aGDvjzFwAWDERERD02Z86c2v379/teuHBBAwBNTU2yYcMGP71erw0LCxvw0xEAr5IgIiLqsRkzZjS++OKL3z388MMTTSYTDAaDPP7441e//vprbWJiYq2j4+sNLBiIiGhwsfMyyN62YsWKaytWrLjmiGP3B05JEBERkU0OKRhE5A0ReVBE/iAia9u1jxeRb0XER0QyRCTYEfFZ0y7msSJyusNPnSUPEflfEfF1dLxERES9qd8LBhG5B8CPlVJ/A/B7AI+KyL3mzRsBrFZKXQfwNoB/6ea++2SKpX3MSqkLSqkoyw+AXwKoBfBf5u7vAVjeF3EQERE5SpcFg4h4iMgBEdGLSJGI/NzcPkVEjohInoh8IiIjRUQjIidF5CFzn/8QkT9Y2e1jAP4CAEqpJgDPA3hTRBIB6JRS75v7fQEg3lYRICKfm7/95wL4jYjMFZHjIpJv/rY/3NzPU0TeFZFCESkQkcfM7QkikiMip0TkAxHx7CrmDsd2A5AOYIVSqtLcnAVgQVcxExERDTS2RhhmAfhOKRWplAoH8BcRcQGwGUCKUmoKgHcA/EEpZQCwGMDbIhJvfu2/WtnnTwDcPCFFKXUQQA2AP6HdN3OllAlAGYBIO/IYopSKUUptAHAUbaMB9wHIwP+PUqwGUKuUilBKTQbwmYj4AfgdgHilVDSAXAC/tRVzO+sAHFVKZbWLuwaAq3lU4hYi8k8ikisiuca6BjvSIiIiujPYGsIvBLDBPD//Z6XUFyISDiAcQLaIAIAzgMsAoJT6SkTeA/BnANOUUrfdIhPASABVHdreBKBVSpV0aP8ewChY/7Bub0+7x2MA7BGRkQCGADhnbo8H8ISlk1KqRkTmAAgD8HdzLkMA5NgTs4jMNu/T2iInlrivtm9USm0FsBUA3ALHKRs5ERER3TG6LBiUUt+ISDSARAC/F5FPAXwE4Cul1LROXhYB4DqAYZ1sbwLg1qHNZP7pyM3c35Yf2j3eDOCPSqks8/TImi5eJwCylVK2phBuiVlEhgFIA5Bsnlb5R+MmIiIaEGydwzAKQKNSaheA1wFEAygB4C8i08x9XERkkvnxPABDATwIYLOI+FjZ7dcAguyMbyKAIvO+d4rIVDte4w3gkvnxr9q1ZwNY0S43XwBfAviJiASZ2zxEZKIdMb8DYLNSKr9jR2kbqhgBoNyOWImIiAYEW1MSEQBeFxETgFYAy5RSLSKSAmCTiHib9/GGiFwB8J8AfqqUuiAiW9B21cOvOuzzAIBnAGzv6sDmkxWb2p1MOBnAd3bktAbAByJSA+AzABPM7b9H28mVRQCMAP5VKfU/IrIYwG4RcTX3+x2AbzqL2VwoPQJgrIgsatcnWym1Cm1TFF+az+kgIqJ+VhT9gLWp4n9Y+Kljdt0IaunSpWOGDRvW+tprr10BAJPJBG9v76hLly4VpKWlDc3Pz/dIT08/39DQIDNmzJiYk5NTotFosG7dOn+9Xq99//33KwAgNTV1VEVFhWtmZua5jsc4deqU29y5c4MrKioKnZ2dYTAYEBcXF3z+/Hm3s2fPFrm6uioASElJCSgqKnK3vK66utqltbVVrly5op8+ffrEnJycEhcXl269D7amJD4B8ImV9tNoG0XoaGK7Pps62ecX5isofMyXT0Ip9TmAzzt0XYi2YX+IiBeAUqXURSv7e6jD848BfGylXwNuL16glPoMQKy1WDuJOQdtUxmd+SWAt7raHxERDT7FxcXahISEOsvzM2fODBk6dKjBy8vLVFhY6B4REdEIAJs3b/ZLSkqq0WjaPoKXLVt2NTg4OLy6uvpSdna2Z3Z2ts/Jkye/tnaM6Ojo5sDAwOaMjAzvRYsW1aampo4OCgpqDgwMvJGdne05Z86cegDYt29fefs44uLiQjdu3Fju5uam4uLi6rZv3z502bJl3borpaPu9LgSwDgbfa6j7coJKKXqlFKP93lUXbMnZgAoUkp92tfBEBHRnaWkpEQbExNzc2VK80qVTUBbMREVFdUEAHv37r1n/vz51y39dDqd6dFHH732/PPPj161atW4ffv2nfX09FQAEBcXF1ReXn7LUMBzzz13JS0tbdiOHTt8jh8/7rl169aLiYmJtQcPHvTqGNPly5c1s2bNmvjCCy98t2jRoloASElJuZ6RkTG0u/k5pGBQSh1XShXY6PPunTSsb0/M5n7b+iMeIiK6c1RVVTkbDAYZN27czc8tvV6vDQ8PbwSA0tJSbWxsbFNzc7NcuHDBNSQk5JarCJ955pnqXbt2+a9du7Zi0qRJNyztR44cKQsICGht33fevHl1lZWVLmvWrBnz0UcfnXV1dVWPPPJI/dGjR3Xt+9XX1zvNnj07KDk5+dqqVauqLe2xsbFNBQUFHt3NkWtJEBER9VBubq42JCTklqvjioqKtJGRkU1lZWUuWq3W5OfnZ6ysrNTodLrbvgy/8sorI319fQ0Gg6GrKe+bYmJifli+fPkVS4Gi0+lMXl5eRstohMFgQHJy8o+CgoKaN27ceMv5fxqNBi4uLqqmpqZbNQALBiIioh4qLi52CwgIuDkyYDQakZeX55mQkFBvnppoBAAPDw9TS0vLLZ+9r7766vAbN2447dy589stW7YMt+d4JSUlblOmTGls3xYfH1+XmZnpBQBPPvnkeIPBILt37y639vrW1lZxd3fv1v2AWDAQERH10IQJE1pOnz7t3tjYKADw0ksvjZw6dWrDiBEjjHq9XhsWFtYEAP7+/kaj0SiWfllZWbr09HS/PXv2nJszZ059Q0OD87Fjx7SW/U6bNm3iuXPnbrucoaysTBsTE3PLiEZSUlLt4cOHvVeuXDmyoKDAff/+/WetXQlRWVnp7OPjY7BcUWGvPlmsiYiIyFHsvQyyN82bN6/u0KFDXsHBweEuLi4qOjq6YceOHeeBtqmJxMTEWkvfBx98sPbw4cOekyZNurF8+fKArKysUl9fXxMALFu27MqGDRuGP/DAA+VGoxHnz5939ff3v2UKo6yszEWn0xm9vb1vueFhdHR08xdffOF16NAh31GjRrXcf//9oZZtgYGBzQcOHPgWAA4dOuQVHx9fi25iwUBERNQLtm3bdhHAbZf/Z2Vl3XI/hdTU1Kr169cPz8zMPHfx4sXCDtuupqamXgXa7rmQmJhYY7liwiIoKKi14+ss5s6de23x4sVXExISfrC2HQB27949dP369bfFaQsLBgcJ9x2G3JRnHR3GnYvvDRENUtOnT2/Mzc2tMxgMsNyLwZrY2Njm2NjYbn2wp6enV3S1vbm5WZKSkq5Pnjz5Rlf9rBGluAaSI4hIPdpus3238gNQbbPX4MX879787+bcgZ7nP14p5d++Qa/Xl0dGRt7N72mv0uv1fpGRkQEd2znC4DglSqkYRwfhKCKSy/yZv6PjcIS7OXeA+Q9kvEqCiIiIbGLBQERERDaxYHCcrY4OwMGY/93tbs7/bs4dYP4DFgsGB1FK3dX/0zB/5u/oGBzlbs4dYP4DGQsGIiIisokFQx8TkVkiUiIiZSLyopXtriKyx7z9uIgE9H+UfceO/H8rIsUiUiAin4rIeEfE2Rds5d6u32MiokRkUJ05bk/+IjLf/Pv/SkTS+zvGvmTH3/44EfmriOSb//4THRFnXxCRd0TkexEp6mS7iMgm83tTICLR/R0jdR8vq+xDIuIM4E0AD6Pt7l8nRSRLKVXcrtvTAGqUUkEi8gSAtQB+3v/R9j47888HEKOUahSRZQDWYRDkb2fuEBEdgN8AON7/UfYde/IXkWAALwH4iVKqRkSGOSba3mfn7/93APYqpd4WkTAABwEE9HuwfWMHgC0AdnayfTaAYPPP/QDeNv+3VzwW9dSU3toXAHx4+l27bjW9dOnSMcOGDWt97bXXrgCAyWSCt7d31KVLlwrS0tKG5ufne6Snp59vaGiQGTNmTMzJySnRaDRYt26dv16v177//vsVAJCamjqqoqLCNTMz85y146SkpAQUFRW5W55XV1e7tLa2yujRo2/k5+efcXV1VadOnXKbO3ducEVFRaGzszMMBgPi4uKCFyxYcG3Hjh1+OTk5JdbWmegKRxj61lQAZUqpb5VSLQAyACR36JMM4E/mx/sA/FRE7FredACwmb9S6q9KKcuKa18CGNPPMfYVe373APBvaCsSm/szuH5gT/5LALyplKoBAKXU9/0cY1+yJ38FwMv82BvAdxgklFJ/A3Ctiy7JAHaqNl8C8BGRkf0TXd8pLi7WRkVF3VwQ6syZM0OGDh1q8PLyMhUWFrpHREQ0AsDmzZv9kpKSaix3eVy2bNnVTz/91Ke6utp59+7d3tnZ2T67du0q7+w4+/btKz9z5kzxmTNnijMzM8ucnZ3VW2+9dS4mJuaH7OxsT6BtXYnAwMDmjIwMbwBITU0dHRQU1Pzss89ejYuLq9u+ffvQ7ubHgqFvjQZwod3zi+Y2q32UUgYAtQDu6Zfo+p49+bf3NIBDfRpR/7GZu3kYdqxS6kB/BtZP7PndTwQwUUT+LiJfisisfouu79mT/xoAvxCRi2gbXfjn/gntjtDdfxsGhJKSEm1MTMzNJafNy1o3AbcWE3v37r1n/vz51y39dDqd6dFHH732/PPPj161atW4ffv2nbWsHxEXFxdUXl5udSjg8uXLmlmzZk184YUXvlu0aFFtYmJi7cGDBy1FKJ577rkraWlpw3bs2OFz/Phxz61bt14EgJSUlOsZGRksGGhgEpFfAIgB8LqjY+kPIuIE4I8AVjo6FgfSoG1I+iEACwBsExEfh0bUvxYA2KGUGgMgEcB75r8LGoCqqqqcDQaDjBs37ubKknq9XhseHt4IAKWlpdrY2Nim5uZmuXDhgmtISEhL+9c/88wz1bt27fJfu3ZtxaRJk26u83DkyJGygICA1o7Hq6+vd5o9e3ZQcnLytVWrVlUDwCOPPFJ/9OhRnaXPvHnz6iorK13WrFkz5qOPPjprWc46Nja2qaCgwKO7OfKPs29dAjC23fMx5jarfUREg7ahyav9El3fsyd/iEg8gJcBJCmlur0gyh3KVu46AOEAPheRcgA/BpA1iE58tOd3fxFAllKqVSl1DsA3aCsgBgN78n8awF4AUErlAHBD2zoLdwO7/m0YSHJzc7UhISFN7duKioq0kZGRTWVlZS5ardbk5+dnrKys1Oh0OkPH17/yyisjfX19DQaDweaUtMFgQHJy8o+CgoKaN27ceHMqS6fTmby8vIztRyRiYmJ+WL58+ZX2hYxGo4GLi4uqqanpVg3AgqFvnQQQLCITRGQIgCcAZHXokwXgV+bHKQA+U4NnRTCb+YvIfQDS0FYsDKY57C5zV0rVKqX8lFIBSqkAtJ2/kaSUynVMuL3Onr/9TLSNLkBE/NA2RfFtfwbZh+zJvwLATwFARO5FW8FQ1a9ROk4WgCfNV0v8GECtUuqyo4PqieLiYreAgICbX3iMRiPy8vI8ExIS6s1TE40A4OHhYWppabnls/fVV18dfuPGDaedO3d+u2XLluG2jvXkk0+ONxgMsnv37vKO2+Lj4+syMzNvTkuUlJS4TZkypbFjv9bWVnF3d+/WZw0Lhj5kPifhWQCfAPgabWdEfyUir4lIkrnbfwO4R0TKAPwWQKeX3w00dub/OgBPAB+IyGkR6fiP6oBkZ+6Dlp35fwLgqogUA/grgFVKqUExumZn/isBLBERPYDdABYPli8LIrIbQA6AEBG5KCJPi8hSEVlq7nIQbcVhGYBtAJY7KNReM2HChJbTp0+7NzY2CgC89NJLI6dOndowYsQIo16v14aFhTUBgL+/v9FoNIqlX1ZWli49Pd1vz5495+bMmVPf0NDgfOzYMa1lv9OmTZt47ty5myMGK1euHFlQUOC+f//+s9auckhKSqo9fPiwt+V5WVmZNiYm5paRj8rKSmcfHx+DZYrCXlzemoiIBrQ7ZXnrJUuWjDl48KCvi4uLio6ObkhLS7vg7+9vTEpKmpCYmFi7dOnSawAwf/788QsXLrw2adKkGzNmzAjJysoqjY6ObgaATZs23XPkyBHdhx9+WG40GjF27NiIb775psjT01OVlJQMCQ0NjRg1alSLTqczWo4bGBjYfODAgZujc6GhoWFFRUXF5eXlLg899FDoxYsXC9vH+e677/oeO3bMY9u2bRet5dHZ8tYsGIiIaEC7UwoGex09etR9/fr1wzu7z4LFyZMn3dLS0vy2b99u9YO9MwsXLhy3ePHiqwkJCT9Y256QkBC4fv36i5MnT7Z6zlhnBQNv3ERERNSPpk+f3pibm1tnMBhguReDNbGxsc2xsbHdKhYAID09vaKzbc3NzZKUlHS9s2KhKxxhICKiAW2gjTDc6TobYeBJj0RERGQTCwYiIiKyiQUDERENdCaTyTRY1uBxKPP7aLK2jQUDERENdEVVVVXeLBp6xmQySVVVlTcAq8uS8yoJIiIa0AwGw68rKyu3V1ZWhoNfhHvCBKDIYDD82tpGXiVBRERENrESIyIiIptYMBAREZFNLBiIiIjIJhYMREREZBMLBiIiIrLp/wCb1dYEnDUyAQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] @@ -1055,11 +1055,11 @@ " {\n", " \"variables\": [\"sex\", \"race\"],\n", " \"metrics\": {\n", - " \"mi\": dit.shannon.mutual_information(q_SGR, 'S', 'RG'),\n", - " \"si\": si_SGR, \n", - " \"ci\": ci_SGR,\n", - " \"ui_0\": ui_SG_R,\n", - " \"ui_1\": ui_SR_G,\n", + " \"mi\": dit.shannon.mutual_information(q_XGR, 'X', 'RG'),\n", + " \"si\": si_XGR, \n", + " \"ci\": ci_XGR,\n", + " \"ui_0\": ui_XG_R,\n", + " \"ui_1\": ui_XR_G,\n", " }\n", " },\n", "])" @@ -1069,7 +1069,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "From the figure above, we can see that `sex(Y)` contains substantial amount of information about `income(S)`, while `race(Z)` does not. Futhermore, the two attributes `sex(Y)` and `race(Z)` also share some information about `income(S)` but not in a complementary manner." + "We see that race (Z) conveys no unique information about income (X) w.r.t sex (Y). " ] }, { @@ -1084,9 +1084,7 @@ "- education and race\n", "- race and occupation\n", "- age-group and sex\n", - "- hours-per-week-group and occupation.\n", - "\n", - "We first define a function that computes the decomposition for us." + "- hours-per-week-group and occupation." ] }, { @@ -1118,7 +1116,7 @@ " si_SXY = si_SXY_1\n", " \n", " ci_SXY = si_SXY - dit.multivariate.coinformation(P, rvs)\n", - " i_S_XY = dit.shannon.mutual_information(P, 'S', to) \n", + " i_S_XY = dit.shannon.mutual_information(P, 'X', to) \n", "\n", " # sanity check\n", " assert math.isclose(i_S_XY, si_SXY + ci_SXY + ui_SX_Y + ui_SY_X, abs_tol=1e-6), \\\n", @@ -1136,7 +1134,7 @@ " }\n", " }\n", "\n", - "decomp_S_HO = information_decomposition(dist_census, 'S', 'HO')" + "decomp_X_HO = information_decomposition(dist_census, 'X', 'HO')" ] }, { @@ -1145,7 +1143,7 @@ "metadata": {}, "outputs": [], "source": [ - "decomp_S_EG = information_decomposition(dist_census, 'S', 'EG')" + "decomp_X_EG = information_decomposition(dist_census, 'X', 'EG')" ] }, { @@ -1154,7 +1152,7 @@ "metadata": {}, "outputs": [], "source": [ - "decomp_S_ER = information_decomposition(dist_census, 'S', 'ER')" + "decomp_X_ER = information_decomposition(dist_census, 'X', 'ER')" ] }, { @@ -1163,7 +1161,7 @@ "metadata": {}, "outputs": [], "source": [ - "decomp_S_RO = information_decomposition(dist_census, 'S', 'RO')" + "decomp_X_RO = information_decomposition(dist_census, 'X', 'RO')" ] }, { @@ -1172,27 +1170,17 @@ "metadata": {}, "outputs": [], "source": [ - "decomp_S_AG = information_decomposition(dist_census, 'S', 'AG')" + "decomp_X_AG = information_decomposition(dist_census, 'X', 'AG')" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, - "outputs": [], - "source": [ - "# didn't converge, should we remove it?\n", - "# decomp_S_EO = compute_decomposition_from(dist_census, 'S', ['E', 'O'])" - ] - }, - { - "cell_type": "code", - "execution_count": 35, - "metadata": {}, "outputs": [ { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAooAAACnCAYAAACW2X1NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de5zVVb3/8dcbBmW4DIog4gVREAhBEAbMNMkkUVJAQ80s0zp1LM1jqaUnLbNfF9NjGWbejmIWkuIlzFQ8mmim4HAZGBEEdVBJFBO5CAgz8/n9sdfGzXYPe9Rhhsv7+Xjsx3y/67u+a32+azb6mbW+370VEZiZmZmZ5WvR3AGYmZmZ2dbJiaKZmZmZFeRE0czMzMwKcqJoZmZmZgU5UTQzMzOzgpwompmZmVlBJc0dgNmOplOnTtG9e/fmDsPMbJsyY8aMtyKic3PHsaNxomjWxLp3705FRUVzh2Fmtk2RtLi5Y9gReenZzMzMzApyomhmZmZmBTlRNDMzM7OCnCiamZmZWUF+mMWsiVUtf5M+k65t7jB2KPMn/bS5Q9jhVL3Qo7lDsO3Ej+t6c/fsW5s7jB2WZxTNzMzMrCAnimZmZmZWkBNFMzMzMyvIieJ2RtIZkhr1BjhJYyT1zdm/XNLwRmr7YEn/K+lASS9IKs059oCkUyUdJ+nyxujvI8QnSY9JKpN0gqTZea86ScdK6izpoeaI0czMbEtxomgNMQbYmChGxI8i4v8aqe3/Bn4bEc8B9wA/hExyCrSKiDuAB4DjJbVppD4/jJFAZUSsjIh7I2Jg9gVcBzwJPBwRy4DXJR3WDDGamZltEU4UtyGSvixpeprJukFSy1R+ZpqNmw4cllN/vKSxOfurc7Z/IGmupEpJv0xl35D0bCq7W1IbSZ8CRgFXpn575LYr6ShJs1Jbt0jaOZVXS/qJpJnpWJ8C19MeOCgiKlPR5cBJkgYCvwTOBoiIAB4HjisyPgfmjM8cSQfUN26S9pW0UFInSS0kPSnp6ALNngb8pUBfvYAfAV+JiLpUfF+qb2Zmtl1woriNkPQJ4BTgsDSbVQucJqkr8BMyCeLh5Mz8baatY4HRwCERMQD4VTp0T0QMSWXPA1+PiH8Ck4EL00zaiznttAbGA6dERH8yH7f0rZyu3oqIQcDvgQsKhFIOVGV3ImJNqvcEMDEiFubUrQA+XeTSzgKuSeNTDrxW37hFxGLgihTb+cC8iJhSoM3DgBm5BZJaAROA8yPilYbEKOmbkiokVdSuXF2oipmZ2VbHieK24yhgMPCspNlpf3/gEODxiFgWEeuBPzegreHArSkxIyLeTuX90szaXDIzYwcWaac38HJEvJD2bwOOyDl+T/o5A+he4PyuwLLcgoi4H3iHzLJurjeBPYvE8zTw35J+AOwbEWupf9yIiJuBMjIJZqFEFqBjRKzKK/sp8FxE5I91vTFGxI0RUR4R5S3L2hW5DDMzs62DP3B72yHgtoi4eJPCzL189akh/TEgqQWwU5E+xgNjIqJS0hnAZz5qsMl76Wcthd9ra4HWBcrr0itX61S/XhExQdI04PPA3yT9J/WMG0C653HvtNsOyE8IAWoktcguL0v6DPAFYFCBukVjNDMz25Z4RnHb8SgwVtLuAJI6StoXmAYMk7RbWhI9KeecajKzaZC5z7BV2n4EODP7cIikjqm8PZkHMlqx6b12q9KxfAuA7pJ6pv2vAFM/xDU9D/QsWiujF2mZOj19/Iv8CpL2B16KiN+Sua/wIOofN8gsPf+JzL2GN9XT7wLSDKSkXYFbgdMLzDJuEqOZmdn2wIniNiIi5gGXAFMkzSGT7HWNiNeBy8gsuz5FJvnKuolMElkJHAq8m9p6iMx9hxVpOTa77HopmcTzKWB+TjsTgQvTQysbv5crItYBZwJ3peXqOuD6D3FN84EO6aGWYo4k8/QzQA9gZYE6JwNV6Zr6AX+ob9wkDQOGAFdExJ+A9ZLOLNDmA7w/s3oWsDvw+7yPyDmlQIxmZmbbPGUeKDVrHpK+C6xK9wvWV6cLMCEijkr7fwS+mz6SZkvH15VMwvm5BtR9AhgdEcs3V691j27R/YrvN1aI1gD+ruem5+96tsaS/a5nSTMiory549nReEbRmtvvef9exvp0I/NkMgAR8eWmSBJTX68DN0kq21w9SZ2Bq4sliWZmZtsSzyiaNbHy8vKoqKho7jDMzLYpnlFsHp5RNDMzM7OCnCiamZmZWUFOFM3MzMysICeKZmZmZlaQE0UzMzMzK8iJopmZmZkV5ETRzMzMzApyomhmZmZmBTlRNDMzM7OCnCiamZmZWUElzR2AmZmZ2UcxY8aM3UtKSm4G+uHJr4+qDqiqqan5j8GDB7+Zf9CJopmZmW2TSkpKbt5jjz0+0blz5+UtWrSI5o5nW1RXV6dly5b1Xbp06c3AqPzjzr7NzMxsW9Wvc+fOK50kfnQtWrSIzp07ryAzK/sBnlE0a2JVy9+kz6Rrm6y/+ZN+2qjtVb3Qo1HbMzMr5sd1ves71MJJ4seXxrDg5KFnFM3MzMysICeKZmZmZlaQE8VtmKQzJDXqGqakMZL65uxfLml4I7V9sKT/lXSgpBckleYce0DSqZKOk3R5Y/TXGLIxp+1bJc3OeVVLeiMdO0fS15o3WjMzs8blexQt3xjgr8A8gIj4USO2/d/A/4uI5yTdA/wQuETSGKBVRNwhScBPJf0yItY0pFFJJRFR04hxfiBmgIg4M6fPFsDjwB9S0S3AU+mnmZk1gz6Trh3cmO3NH3vOjGJ1Fi9e3Orss8/ep7q6eufVq1e37NKly/pp06a9cOWVV3aaNWtW2wkTJixuzJiammcUt1KSvixpepq5ukFSy1R+ZpqNmw4cllN/vKSxOfurc7Z/IGmupEpJv0xl35D0bCq7W1IbSZ8i82j8lanfHrntSjpK0qzU1i2Sdk7l1ZJ+ImlmOtanwPW0Bw6KiMpUdDlwkqSBwC+BswEiIsgkYMcVGZ8zJE2W9BjwqKR2kh7NiWF0Tt3TJc1J13p7KuucrvvZ9DqsQB/5Mef6b2BZRNyc4l4DVEsaurm4zcxs+3LqqafuN3r06OVVVVXPV1dXV40bN+5VgLlz57bp379/gyY8tmZOFLdCkj4BnAIcFhEDgVrgNEldgZ+QSRAPB/rW38rGto4FRgOHRMQA4Ffp0D0RMSSVPQ98PSL+CUwGLoyIgRHxYk47rYHxwCkR0Z/MbPS3crp6KyIGAb8HLigQSjlQld1JidUFwBPAxIhYmFO3Avh0sWsDBgFjI2IYsA44IcVwJPA/yjgQuAT4bLrW/0rnXgP8OiKGAF8Abi4Wc1ZKBv8D+EbeoXrjlvRNSRWSKmpXri5UxczMtjE1NTVMnz69/YgRI1Zly4YOHboWYN68eaUDBw5c23zRNQ4niluno4DBwLOSZqf9/YFDgMcjYllErAf+3IC2hgO3ZpdxI+LtVN5P0pOS5gKnAQcWaac38HJEvJD2bwOOyDl+T/o5A+he4PyuwLLcgoi4H3gHuC6v7pvAnkXiAXgk53oE/FzSHOD/gL2ALsBngbsi4q3UZ7b+cODaNL6TgTJJ7YrFnOr8kUxi/XZe/XrjjogbI6I8IspbluV3Y2Zm26KSkhIOPfTQlQcffPCBX/rSl/adMmVK2+yxhQsXlg4ZMsSJom0RAm5Ls3oDI6J3RFxW5Jwa0u8z3T+3U5H644Fz0uzgT4DWHy9k3ks/ayl87+vaevqoS69crVP9Yt7N2T4N6AwMTrOwb9TTX1YL4JM5Y7xXRORP9RWKeRzwl4h4tECbDY3bzMy2E1OnTl04YcKEF8vKympPOOGEXrfffvsuixYtalVaWlrXqVOn2uaO7+Nyorh1ehQYK2l3AEkdJe0LTAOGSdpNUivgpJxzqsnMQkLmPsNWafsR4ExJbbJtpfL2wOupndNy2lmVjuVbAHSX1DPtfwWY+iGu6XmgZ9FaGb1IS76STpD0iwac0wF4MyI2SDoS2DeVP0bmXsjdUnvZ658CfCd7crpXcrMxp3s1B5B5CGezcZuZ2Y6hRYsWjBgxYvX111//2vHHH/92ZWVl6YwZM9r06dNnm78/EZwobpUiYh6Z++qmpKXUR4CuEfE6cBnwNJknbJ/POe0mMklkJXAoabYtIh4is7RakZZZs/cPXkom8XwKmJ/TzkTgwvTQysav4IiIdcCZwF1puboOuP5DXNN8oEN6QKSYI4EH0nYPYGUDzvkTUJ5iO510TRHxHPAzYGoam6tT/XNT/TmS5gFnNSDmn5GZtcw+ZJR9ZT/m5zAyvyszM9sB3H333WXr1q0TwJIlS0qmT5/e7phjjllZWVlZ2rdv3+1ihUmZh0zNtjxJ3wVWZZ8UrqdOF2BCRByV9v8IfDciltV3zpbUkJhTvYOB70XEV4q12bpHt+h+xfcbK8Si/BV+Zrat+3Fdb+6pHD8jIspzyysrK6sHDBjwVnPF9cUvfnHfp556qn2bNm3qdtppp/je97639Mwzz1w+atSo/UaOHLnirLPOyr+XfatVWVnZacCAAd3zy/05itaUfs+my+WFdAPOz+5ExJe3aETFNSRmgE5kZmnNzGwHMXHixIKfkTh58uSXmzqWLcWJojWZtHx9e5E6zzZROA3SkJhTvQYvOffbdXcqxp7zseL6UBq5r36N2pqZWXF3A9L45g5jh+R7FM3MzMysICeKZmZmZlaQE0UzMzMzK8iJopmZmZkV5ETRzMzMzApyomhmZmZmBTlRNDMzM7OC/DmKZmZmtn34YpfBjdrexDdmNKTaLbfcsutvfvObLuvXr29RU1Oj0aNHv73HHntsmDVrVtsJEyYU/FDubYVnFM3MzMw+onHjxu129dVX73Hvvfe+OH/+/HmzZs16vmPHjrVz585t079//zXNHd/H5UTRzMzM7CN4++23W1x66aX73HnnnS/26NFjA0D79u3rfvjDH745b9680oEDB65t7hg/LieKZmZmZh/BhAkTdh0wYMDqvn37rs8/tnDhwtIhQ4Y4UTQzMzPbEVVVVZX279//A8ngokWLWpWWltZ16tSptjniakxOFM3MzMw+grZt29bW1dV9oHzGjBlt+vTps83fnwh+6tmsyVUtf5M+k65tsv7mT/ppo7b3hfkjG7U9M7Ni7p59a3OHUNBxxx234vTTT9//4osvfmOfffapWbt2ra677rrd/v3vf5f07dt3m192BieKZmZmtr1o4MfZNJYjjzxyzUUXXfSvz33uc73q6uqoqanRSSed9O/nn3++dOTIkSuaMpYtxYmimZmZ2Ud09tlnv3322We/3dxxbCm+R3EHIalU0lRJbSXNl9Q/59iFkm6Q1FnSQ80Z50chaaCkkTn7oyRd1EhtZ8etpaSzJc3OeVVJCkmfkNRf0vjG6NPMzGxr4URxK6KMLfU7+RpwT0S8C5wHXJf62ws4C7goIpYBr0s6bAvFsKUMBDYmihExOSJ+2UhtZ8etNiJ+FxEDsy9gMvCniHg+IuYCe0vq1kj9mpmZNTsnis1MUndJCyT9AagC9pH0e0kVkp6T9JOcukMk/VNSpaTpktqnma4rJT0raY6k/6ynq9OAvwBExEPA68DpwK+ByyJieap3X6pbLO7vpRm1Kknn5ZSfnuKolHR7Kusi6d5UVinpU+m6q3LOu0DSZWn7cUnX5MzaDU3lQyU9LWlWGofeknYCLgdOSfVPkXSGpGtzxvexFNOj2URO0nhJv03tvCRpbLFxy7v+I4CTgW/nFN8PfLHY2JmZmW0rfI/i1uEA4KsR8QyApB9GxNuSWgKPSjoImA/8GTglIp6VVAasBb4OrIiIIZJ2Bp6SNCUiXs42npKp/SOiOqfP84DpwMKIuD2nvAL4f5sLVtJg4EzgEEDANElTgfXAJcCnIuItSR3TKb8FpkbECema2gG7FhmTNhExMCVktwD90hh8OiJqJA0Hfh4RX5D0I6A8Is5J8Z2R08444LaIuE3S11IsY9KxrsDhQB8ys4OT8q6z0LghaRdgPPCViFiZc6gCuAj4VZFrMzMz2yY4Udw6LM4micnJkr5J5vfTFegLBPB6RDwLkE1QJB0NHJQzI9aBTOL5ck57nYB3cjuMiH9Jegz4a14sbwJ7Fon3cODetIyNpHuAT6cY74qIt1If2Zt7P0tm9pKIqAVWSCqWKN6R6j8hqSwlZ+2B2yQdkPpqVaQNgEOBE9P27WyaxN0XEXXAPEldCpz7gXFLrgduj4in8srrHbv0+/wmQEmnYpduZma2dXCiuHV4N7shaT/gAmBIRCxPD0i03sy5Ar4TEQ9vps7aetqoS69crVP9La2GTW99yI8vCuz/FPh7mpnsDjz+MWN4L2dbBY5/YNwkfRXYF/hygfr1jl1E3AjcCNC6R7f8azMzM9sq+R7FrU8ZmcRxRZrlOjaVLwC6ShoCkO5PLAEeBr4lqVUq7yWpbW6D6f7DlpI2l3Bm9SJzrySS9pL0aIE6TwJjJLVJfZ2Qyh4DTpK0Wzo/u/T8KPCtVNZSUgfgDWB3SbulJfPj8vo4JdU/nMzS+goys6VL0vEzcuquIjPbWMg/ef++wdNSnA2SP26S9gd+DpwWETUFTtk4dmZmZtsDJ4pbmYioBGaRuR9vAvBUKl9PJnkaJ6kSeITMDNbNwDxgZno45AYKzxRPIbNkXMyRwANpuyuZmb/8GGeSuUdvOjANuDkiZkXEc8DPgKkpxqvTKf8FHClpLjAD6BsRG8g8hDI9Xcv8vG7WSZpFZpn366nsV8AvUnnuNf4d6Jt9mCWvne8AZ0qaA3wlxfJh5I7bD4A2wD3a9GNyPp2O546dmZnZNk8RXgXbEUgaBHw3Ir5SpN4TwOi07H0O8EpETG6SIN+P4XHggoioaMp+64mloeO2MzAVOLye2caNWvfoFt2v+H4jRrl5/go/M9vW3T37ViTNiIjy3PLKysrqAQMGvJXdrxr0qcGN2W+/mf8s+k0vZ5111t677777hssvv/wNgLq6Ojp06DBwyZIlc2644YaOs2bNajthwoTFq1ev1pFHHtnr6aefXlBSUsLixYtbnX322ftUV1fvvHr16pZdunRZP23atBfy2585c2br448//oBXXnllbsuWLampqWHYsGEHLF68uPWLL75YtfPOO29M5MaOHdu9qqqqTXb/rbfearVhwwa98cYblYcffnivp59+ekGrVoVv76+srOw0YMCA7vnlnlHcQaRZwL+np44LktQZuDr7UTkRcW1TJ4lbm4aMW9KNzGdRbjZJNDOz7cu8efNKBw4cuPH+9Pnz5+/UsWPHmrKysrq5c+e26d+//xqAcePGdRo1atTykpLMgtipp5663+jRo5dXVVU9X11dXTVu3LhXC7U/aNCgdT169Fg3ceLEDgDnnnvuXj179lz32c9+dsUjjzzSLrfupEmTqufPnz9v/vz58+67775FLVu2jOuuu+7l1q1bx7Bhw1befPPNHQv1sTlOFHcgEXFLeuq4vuPLIuK+poypnjg+szXMJmYVG7dUZ2FEPN5EIZmZ2VZiwYIFpeXl5Wuy+zNmzGjTp0+ftbBpEnnnnXfudvLJJ78DUFNTw/Tp09uPGDFiVfa8oUOHrgUYNmxYz+rq6k2m/c4777w3brjhht3Hjx+/y7Rp09rdeOONr40cOXLF3/72t7JCMb3++uslxxxzTK8LLrjgX6eddtoKgLFjx74zceLED50o+qlnsybWb9fdqRh7TtN12Mh93d2orZmZbbuWLVvWsqamRt26ddu4mlRZWVnar1+/NQALFy4sHTJkyNp169bp1Vdf3bl3797rAUpKSjj00ENXHnzwwQeOGDHinTPOOOOto48++l2AqVOnLsrv58QTT1x50UUX7X3ZZZft/cQTT8zfeeed4/Of//yqn//8513z665atarFscce23P06NFvX3jhhRuX5YcMGbJ2zpw5bfPrF+MZRTMzM7OPoKKiorR3796bfCxaVVVV6YABA9YuWrSoVWlpaV2nTp1qly5dWtK+fftNbk2aOnXqwgkTJrxYVlZWe8IJJ/S6/fbbd9lcX+Xl5e9++9vffiOblLZv376urKysNnf2saamhtGjR+/fs2fPdddcc82/cs8vKSmhVatWsXz58g+V+zlRNDMzM/sI5s2b17p79+4bP5O3traWGTNmtDv66KNXpSXoNQBt27atW79+/SY5V4sWLRgxYsTq66+//rXjjz/+7crKytLN9bVgwYLWgwcPXpNbNnz48JX33XffxuXn008/fd+amhrdcccd1YXa2LBhg9q0afOhnmJ2omhmZmb2Eey3337rZ8+e3WbNmjUCuPjii7sOHTp09R577FFbWVlZ2rdv37UAnTt3rq2trVW23t133122bt06ASxZsqRk+vTp7Y455piVAIceemivl19++QOPJi9atKi0vLx8k9nLUaNGrZgyZUoHgPPPP7/rnDlz2tx///0vFnqyeenSpS132WWXmtynpBvC9yiamZnZdqEhH2fTmE488cSVDz74YNkBBxzQr1WrVjFo0KDV48ePXwyZJeiRI0euyNY94ogjVkyZMqXdmDFjVt111127nnfeed3atGlTt9NOO8Wll176r+HDh79bW1vL4sWLd+7cufMmy9SLFi1q1b59+9oOHTps8m1qgwYNWvfSSy+1fu6553a++uqr99xzzz3XH3LIIX2yx3v06LHugQceeAngwQcfLBs+fPgKPiQnimZmZmYf0U033fQa8Fp++eTJk1/O3T/33HOXXXXVVV3GjBmzauLEiYsLtTVz5szWI0eOXN6uXbtNZv169uy54bXXXptb6JxBgwatXrJkSUlEbDZJvuOOOzpeddVVH4izGCeKZmZmZlvY4YcfvqaiomJlTU0N2c9SzDdkyJB1Q4YM+VDJ3IQJE14pVmfdunUaNWrUOwcddNB7xermc6JoZmZm1gTOO++8fzdHv61bt45zzjnnI/Xth1nMzMzMrCAnimZmZmZWkBNFMzMzMyvIiaKZmZmZFeRE0czMzMwK8lPPZk2savmb9Jl0bXOHYR/C/Ek/be4QdihVL/Ro7hBsK9Nv5j+bO4QdlmcUzczMzKwgzyiamZnZduELA88c3Jjt3T371qJfCXjWWWftvfvuu2+4/PLL3wCoq6ujQ4cOA5csWTLnhhtu6Dhr1qy2EyZMWLx69WodeeSRvZ5++ukFJSUlLF68uNXZZ5+9T3V19c6rV69u2aVLl/XTpk17oVAfY8eO7V5VVdUmu//WW2+12rBhg/baa6/3Zs2aNT/7/c0zZ85sffzxxx/wyiuvzG3ZsiU1NTUMGzbsgFNPPfXt8ePHd3r66acXFPoe6M3xjKKZmZnZRzRv3rzSgQMHrs3uz58/f6eOHTvWlJWV1c2dO7dN//791wCMGzeu06hRo5Znv5Xl1FNP3W/06NHLq6qqnq+urq4aN27cq/X1MWnSpOr58+fPmz9//rz77rtvUcuWLeO66657uby8/N1HHnmkXbbeoEGD1vXo0WPdxIkTOwCce+65e/Xs2XPdOeec8+9hw4atvPnmmzt+2OtzomjNStJ5kk6X9A1Jf84pL5P0oqT9JV0l6bPNFF9XSX9N2z+TNDvn9YKkWkntJB0n6fLmiNHMzJrPggULSsvLy9dk92fMmNGmT58+a2HTJPLOO+/c7eSTT34HoKamhunTp7cfMWLEqux5Q4cOXQswbNiwntXV1QWn/V5//fWSY445ptcFF1zwr9NOO23FyJEjV/ztb38ry61z3nnnvXHDDTfsPn78+F2mTZvW7sYbb3wNYOzYse9MnDjRiaI1Lklb7PaE1PbXgAnAzcA+koanw5cDt0TES8A44KItFUcR3wNuAoiIH0bEwOwLeBb4RUSsBh4AjpfUZjNtmZnZdmTZsmUta2pq1K1bt5psWWVlZWm/fv3WACxcuLB0yJAha9etW6dXX3115969e68HKCkp4dBDD1158MEHH/ilL31p3ylTprTNnj916tRF3bt335Df16pVq1oce+yxPUePHv32hRde+BbA5z//+VX/+Mc/2ufWO/HEE1cuXbq01WWXXbb3vffe+2J2WXrIkCFr58yZ0za/3WKcKG4DJN0naYak5yR9M6f862lWa7qkmyRdm8o7S7pb0rPpdVg97Y6UND+1/ducmbPLJN0u6SngdkmtJd0qaa6kWZKOTPXOyPaZ9v8q6TNpe7WkX6eYH5XUuUAInwVmRkRNRARwFvAbSeXAUcCVABGxGNhN0h5FxukkSVWSKiU9kcpaSroyjcMcSf+Zyk9IcSnNGr5QT/tfAB4q0NeXgZ7AZSnGAB4HjttcjGZmtv2oqKgo7d2799rcsqqqqtIBAwasXbRoUavS0tK6Tp061S5durSkffv2Nbn1pk6dunDChAkvlpWV1Z5wwgm9br/99l3q66empobRo0fv37Nnz3XXXHPNv7Ll7du3rysrK6vNn4EsLy9/99vf/vYbuQlsSUkJrVq1iuXLl3+o3M+J4rbhaxExGCgHzpW0m6Q9gUuBTwKHAX1y6l8D/DoihpBJdG7Ob1BSa+AG4NjUdn4i1xcYHhGnAmeTyYX6A6cCt6XzN6ctUBERBwJTgR8XqHMYsPFG4YiYAzwMPAp8JyLW59Sdmepvzo+AERExABiVyr4OrEhjMQT4hqT9IuJe4PV0bTcBP46IpbmNSdoPWB4R7+WVdwd+CZwWEbn/8CuATxcKTNI3JVVIqqhdubrIZZiZ2bZg3rx5rbt3777x/xG1tbXMmDGj3dFHH70qLUGvAWjbtm3d+vXrN8m5WrRowYgRI1Zff/31rx1//PFvV1ZWltbXz+mnn75vTU2N7rjjjur8Y8OHD1953333bbL8vGDBgtaDBw9ek193w4YNatOmTXyYa3SiuG04V1Il8AywD3AAMBSYGhFvR8QG4K6c+sOBayXNBiYDZZLa5bXZB3gpIl5O+3fkHZ8cEdm/kg4H/ggQEfOBxUCvIjHXAdl7Dv+Y2sjXFViWV/Y7YElEPJ5X/iawZ5E+nwLGS/oG0DKVHQ2cnsZiGrAbmfED+A5wMfBeRORff8H4JLVM13NpRCxqaIwRcWNElEdEecuy/F+FmZlti/bbb7/1s2fPbrNmzRoBXCgxuScAABGzSURBVHzxxV2HDh26eo899qitrKws7du371qAzp0719bW1ipb7+677y5bt26dAJYsWVIyffr0dsccc8xKgEMPPbTXyy+/vHGG8Pzzz+86Z86cNvfff/+LhZ5YHjVq1IopU6Z0yC1btGhRaXl5+SYznUuXLm25yy671GSXohvKH4+zlUtLucOBQyNijaTHgWKzeS2AT0bEury2Hga6kJn5KvaJz+82ILwaNv1jY3NxFXpjri1wTl165Wud6tffQcRZkg4BPg/MkDQYEJnZyYcLnLJ36quLpBYRkd9vofguAV6PiFs/SoxmZrblNOTjbBrTiSeeuPLBBx8sO+CAA/q1atUqBg0atHr8+PGLIbMEPXLkyBXZukccccSKKVOmtBszZsyqu+66a9fzzjuvW5s2bep22mmnuPTSS/81fPjwd2tra1m8ePHOnTt3rgFYsGDBTldfffWee+655/pDDjlk48phjx491j3wwAMvQeZJ55deeql1TU0NJSUlLFq0qFX79u1rO3TosMn/0x588MGy4cOHr+BDcqK49etAZvlzjaQ+ZJaaIfMgxW8k7QqsIrPEPDcdm0JmtuxKAEkDI2J2RIzINiqpFNhfUveIqAZO2UwMTwKnAY9J6gV0AxYAZcC3JbUA9iIzy5nVAhgLTAS+BPyjQLvPk7nPryF6kWZNJf0CmJ6WjzeS1CMipgHTJB1LZvb1YeBbkh6LiA0p/iXAe8AtZJbSv0rmoZWr8vp8Aeie0/4ngTOAQZuJsaqB12NmZtuBm2666TXgtfzyyZMnv5y7f+655y676qqruowZM2bVxIkTFxdqa+bMma1Hjhy5vF27dgHQu3fv9RFRNPkdNGjQ6scee6zt0Ucf/W7Pnj03vPbaa3Pz69xxxx0dr7rqqg/EWYwTxa3fQ8BZkp4nk5w9AxARSyT9HJgOvA3MB7J/KZwL/E7SHDK/4yfIPCiyUUSslfRt4CFJ75JJPOtzHfB7SXPJzCKeERHvpYddXgbmkUn6Zuac8y4wVNIlZJZkCyWiDwK3FxsASa3IJJQVqag/mSX1fFdKOoDMLOKjQCUwh0yyN1OSyCwljwHOB56MiH+kZf1nJT0QEc9nG4uId9NH9PRMy8w/AdoAf880tdEXIuJF4EgyS9lmZmabOPzww9dUVFSszM78FTJkyJB1Q4YM+dDJ3IQJE17Z3PF169Zp1KhR7xx00EHvba5eIco8rGnbIkntImJ1+piZe8l8nMy9xc4rcL7I3Bu4MCJ+3UixrY6IojfjSboX+H5ELNxMnROAQRFxadp/OHd2dEtKfQ+OiEuK1OsCTIiIo4q12bpHt+h+xfcbK0RrAv6u56bl73q2fP1m/hNJMyKiPLe8srKyesCAAW81V1zbk8rKyk4DBgzonl/uh1m2bZelhzSqyMzs3fchz/9GOv85MkvcNzRyfA1xEZmHRjanBPif7E5TJYmpr3uB6gZU7UZmltLMzGy74RlFsyZWXl4eFRUVxSuamdlG9cwovtS/f//lLVq0cDLzMdTV1Wnu3Lm7DhgwYP/8Y55RNDMzs21V1bJlyzrU1dWpeFUrpK6uTsuWLetAPQ9j+mEWMzMz2ybV1NT8x9KlS29eunRpPzz59VHVAVU1NTX/UeigE0UzMzPbJg0ePPhN3v8mLtsCnH2bmZmZWUFOFM3MzMysICeKZmZmZlaQE0UzMzMzK8iJopmZmZkV5ETRzMzMzApyomhmZmZmBTlRNDMzM7OCnCiamZmZWUFOFM3MzMysIH+Fn1kTq1r+Jn0mXdvcYVgjmz/pp80dQrP6wvyRzR2Cbcfunn1rc4eww/KMopmZmZkV5ETRzMzMzArabKIoqbukqqYKZlskqVpSp+aOoxhJXSX9VdLuKeY9co79TtLFkvpLGt+MYX4kkj4j6VM5+2dJOr2R2u4q6a9p+2eSZue8XpBUK6mdpOMkXd4YfZqZmW0tmmVGUVKT3BspqWVT9NMYlLElfx/fA26KiDeBXwJXpX4HAZ8GroqIucDekrptwTi2hM8AGxPFiLg+Iv7QSG1/D7gptfvDiBiYfQHPAr+IiNXAA8Dxkto0Ur9mZmbNriGJSUtJN0l6TtIUSaUAkgZKekbSHEn3Sto1lT8uqTxtd5JUnbbPkDRZ0mPAo2mm5ok0M1Ml6dP5Hadz/pLaXCjpxznHvixpejr/hmxSKGm1pP+RVAkcmtfe7ySNStv3SrolbX9N0s+KtHu0pKclzZR0l6R2eW2XSnpQ0jcKXEdnSY+kMbxZ0uI0Nt0lLZD0B6AK2EfSqZLmpjG5IqeN1TnbY7Mzf5LGS7peUkWa4Tqunt/jF4CH0vaNQA9JRwK/A86JiA3p2P3AF+tpI9u/JF2ZYpwr6ZScYz9IZZWSfpnKekr6v1Q2U1KPNAv415zzrpV0RtqulvSr1M50ST1T+fGSpkmaldrrIqk7cBbw3fQ7+7SkyyRdkM7Z3Pv0itT+C4XefwXGLXcMvgz0BC4DiIgAHgfqG38zM7NtTkMSxQOA30XEgcA7ZP7HCfAH4AcRcRAwF/hxPefnGgSMjYhhwJeAh9PMzABgdj3nDE19HgScJKlc0ieAU4DD0vm1wGmpfltgWkQMiIh/5LX1JJnZM4C9gL5p+9PAE/W1q8zS8iXA8IgYBFSQmWnKakcmwbojIm4qcA0/Bh5LYzgJyJ2xOwC4Lh3bAFwBfBYYCAyRNKaeccnVPY3T54HrJbXOPShpP2B5RLwHEBF1wLeAu4EFEfFETvUK3h+j+pyY4hsADAeuTIn/scBo4JCIGAD8KtX/E5n30AAyM3+vN+CaVkREf+Ba4Dep7B/AJyPiYGAi8P2IqAauB36dZvqezGtnc+/TkogYCpxHgfdv/rjllHcnMyt7WkTU5BxqyNiZmZltMxqyBPxyRGSTuBlAd0kdgF0iYmoqvw24qwFtPRIRb6ftZ4FbJLUC7svpo9A5/waQdA9wOFADDAaelQRQCryZ6teSSYAKeRI4T1JfYB6wq6SuZGYezwW+Wk+7nySTVD6VyncCns5p9y/AryLiT/X0ezhwAkBEPCRpec6xxRHxTNoeAjweEcvS9f4JOAK4r552s+5Myd9CSS8Bfdg08e4KLMs9ISJmK3P/6XV5bb0J7Fmkv8PJJMW1wBuSpqbYhwG3RsSa1MfbktoDe0XEvalsXbq2Il1wR87PX6ftvYE/p9/ZTsDLm2ugAe/Te9LPGWSS7XwfGLc0w/xH4NKIWJRXv96xk/RN4JsAJZ123VzYZmZmW42GzCjmzqbUUjy5rMlpt3XesXezG2kW6whgCTBe0umSTtD7DwqUZ6vmtRGAgNty7hfrHRGXpePrUgKDpENy2hsVEUuAXYBjgCfIJI4nA6sjYtVm2hWZhDVb3jcivp4T01PAMUrZj6Szc/otlnS9W+R47nVn5Y9roTHKtbbAOQB16ZWrdaq/peW+T7L95ooC2+OAa9NM438WOOfDyr6363tfFxq3S4DXI6LQh3rVO3YRcWNElEdEecuydoWqmJmZbXU+0sMTEbECWJ5zX9dXgOysTTWZWTmAsfW1IWlf4I20VHszMCgi7s1JxipS1c9J6qjMvZFjyCRljwJjJe2e2uqY2suPc1pOe5NT8TNklhqzieIF6SebafcZ4LCce+XaSuqV09WPgOVk7vcjIn6X0++/Uswnp3OPBuqbUpoODEv3L7YETs0Z1zckfUKZB15OyDvvJEktJPUA9gcW5B1/gcIzZoX0InO/JJKGpvsn8z0JnCKppaTOZBL+6cAjwJlKD3RI6pgS8NeyS+iSdk7HFwN90/4uwFF5fZyS8zM7e9uBzB8WkJn9zVoFtM8Pssj7tCE2GTdJnwTOIM0MFrBx7MzMzLYHH+cp26+SuTdtDpn71bIfDXIV8C1Js4DNfWzMZ4DKVO8U4Jp66k0ns5Q8B7g7IioiYh6ZmZ0pqf9HyCwTNsSTZO5NWwTMBDqmMuprNy0FnwHckcqfJrO8m+u/gFJJv+KDfgIcnZZ6TwKWkkluNhERrwMXAX8HKoEZEfGXdPgi4K/AP/ngPX6vkBmnB4Gzssu7Oe2+C7yYTXSLOJLME7yQuZey0AzZvWR+H5XAY2TuFVwaEQ8Bk4EKSbPJJOGQSdDOTWP3T2CPiHgVuJNMYnUnMCuvj11T/f8CvpvKLgPukjQDeCun7v1AdjY6/x7B+t6nRRUYt58AbYC/a9OPyemRjueOnZmZ2TZPmYc1t07pKdjyiDinuWP5OCTtDNRGRI2kQ4Hfp4dlGqPt8cBfI2JSkXonAIMj4pIicU4FDk+xXgncHhFzGiPWhlLmSfnyiHirWN0miKXouKV6XYAJEZE/M/oBrXt0i+5XfL+xQrSthL/Cz1/hZ1vO3bNvRdKMiCgvXtsak7/ruWl0A+5My8brgQ98hM6WFhH3StqtSLVuwEXZJ3kj4sItH9nWrYHjBpmxO39Lx2NmZtaUtupEMSLGA+ObOYyPLSIWAgdvobbP+BB1by5yfCGw8OPG9HFFRPfmjiFXsXFLdZ5tiljMzMya0la99Gy2PSovL4+KioriFc3MbCMvPTePZvkKPzMzMzPb+jlRNDMzM7OCnCiamZmZWUFOFM3MzMysICeKZmZmZlaQn3o2a2KSVvHBr1ncUXVi02/Z2ZF5LN7nsXifxyKjE9A2Ijo3dyA7mq36cxTNtlML/BEPGZIqPBYZHov3eSze57HISOPQvbnj2BF56dnMzMzMCnKiaGZmZmYFOVE0a3o3NncAWxGPxfs8Fu/zWLzPY5HhcWgmfpjFzMzMzAryjKKZmZmZFeRE0WwLkHSMpAWSFkm6qMDxnSX9OR2fJql700fZNBowFt+TNE/SHEmPStq3OeJsCsXGIqfeFySFpO32adeGjIWkk9N74zlJE5o6xqbSgH8j3ST9XdKs9O9kZHPE2RQk3SLpTUlV9RyXpN+msZojaVBTx7ijcaJo1sgktQR+BxwL9AVOldQ3r9rXgeUR0RP4NXBF00bZNBo4FrOA8og4CJgE/Kppo2waDRwLJLUH/guY1rQRNp2GjIWkA4CLgcMi4kDgvCYPtAk08H1xCXBnRBwMfBG4rmmjbFLjgWM2c/xY4ID0+ibw+yaIaYfmRNGs8Q0FFkXESxGxHpgIjM6rMxq4LW1PAo6SpCaMsakUHYuI+HtErEm7zwB7N3GMTaUh7wuAn5L5w2FdUwbXxBoyFt8AfhcRywEi4s0mjrGpNGQsAihL2x2AfzVhfE0qIp4A3t5MldHAHyLjGWAXSV2bJrodkxNFs8a3F/Bqzv5rqaxgnYioAVYAuzVJdE2rIWOR6+vAg1s0ouZTdCzSMto+EfFAUwbWDBryvugF9JL0lKRnJG1ulmlb1pCxuAz4sqTXgL8B32ma0LZKH/a/KfYx+ZtZzGyrIOnLQDkwrLljaQ6SWgBXA2c0cyhbixIyy4ufITPL/ISk/hHxTrNG1TxOBcZHxP9IOhS4XVK/iKhr7sBs++cZRbPGtwTYJ2d/71RWsI6kEjLLSf9ukuiaVkPGAknDgR8CoyLivSaKrakVG4v2QD/gcUnVwCeBydvpAy0NeV+8BkyOiA0R8TLwApnEcXvTkLH4OnAnQEQ8DbQm893HO6IG/TfFGo8TRbPG9yxwgKT9JO1E5ubzyXl1JgNfTdtjgcdi+/xQ06JjIelg4AYySeL2eh8aFBmLiFgREZ0ionv6TttnyIxJRfOEu0U15N/IfWRmE5HUicxS9EtNGWQTachYvAIcBSDpE2QSxWVNGuXWYzJwenr6+ZPAioh4vbmD2p556dmskUVEjaRzgIeBlsAtEfGcpMuBioiYDPwvmeWjRWRu3P5i80W85TRwLK4E2gF3ped5XomIUc0W9BbSwLHYITRwLB4GjpY0D6gFLoyI7W7WvYFjcT5wk6Tvknmw5Yzt9A9LJN1B5g+ETumezB8DrQAi4noy92iOBBYBa4AzmyfSHYe/mcXMzMzMCvLSs5mZmZkV5ETRzMzMzApyomhmZmZmBTlRNDMzM7OCnCiamZmZWUFOFM3MzMysICeKZmZmZlaQE0UzMzMzK+j/A7CXJhkauzH/AAAAAElFTkSuQmCC\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAosAAACnCAYAAAB5GxZzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de5xVVf3/8debizBcBUHEC46CQFyEYNC8JJlESgpqqJllWt/KW3w1tfSbqdmvi5cs00yUr+IN0FAJMxW/mmhewOEyMCIIKpAmCkoIAsLMfH5/7H3wMJ7DGXGYGeD9fDzOY85ee+21PnvNQT+z1t77KCIwMzMzM8ulUX0HYGZmZmYNl5NFMzMzM8vLyaKZmZmZ5eVk0czMzMzycrJoZmZmZnk5WTQzMzOzvJrUdwBmO5sOHTpEcXFxfYdhZrZdmTFjxoqI6FjfceyMnCya1bHi4mJKS0vrOwwzs+2KpCX1HcPOysvQZmZmZpaXk0UzMzMzy8vJopmZmZnl5WTRzMzMzPLyDS5mdax85bv0nHhTfYexU5k/8Zf1HcJOp/zVrvUdgu0grqjqwQOz76jvMHZqnlk0MzMzs7ycLJqZmZlZXk4WzczMzCwvJ4s7GElnSKrVC+IkHS+pV9b2VZKG1FLbn5f0v5J6S3pVUlHWvkcknSrpWElX1UZ/WxGfJD0lqY2kEyTNrvaqknSMpI6SHquPGM3MzLYlJ4tWE8cDm5LFiLg8Iv6vltr+H+CPEfEy8CDwM0gSVKBpRIwHHgGOk9Silvr8NIYBZRHxQUQ8FBH9My/gZuBZ4PGIWA68LemweojRzMxsm3GyuB2R9C1J09MZrdGSGqflZ6azctOBw7Lqj5U0Mmt7Tdb7n0qaK6lM0m/Tsu9Leikte0BSC0mHAsOBa9N+u2a3K+koSbPStm6X1CwtXyzpF5Jmpvt65jif1sCBEVGWFl0FnCSpP/Bb4FyAiAjgaeDYAuPTO2t85kg6IN+4SdpX0kJJHSQ1kvSspKE5mj0N+GuOvroDlwPfjoiqtHhSWt/MzGyH4WRxOyHpc8ApwGHprFYlcJqkzsAvSJLEw8maAdxCW8cAI4CDI6IfcE2668GIGJSWvQJ8LyKeByYDF6czaq9ltdMcGAucEhF9SR7FdHZWVysiYgDwZ+CiHKGUAOWZjYhYm9Z7BpgQEQuz6pYCXyxwamcBN6TjUwK8mW/cImIJcHUa24XAvIiYkqPNw4AZ2QWSmgLjgAsjYmlNYpT0A0mlkkorP1iTq4qZmVmD5GRx+3EUMBB4SdLsdHt/4GDg6YhYHhEbgPtq0NYQ4I40OSMi3k/L+6QzbHNJZsh6F2inB/BGRLyabt8JHJG1/8H05wygOMfxnYHl2QUR8TDwH5Il3mzvAnsWiOcF4H8k/RTYNyLWkX/ciIgxQBuSJDNXMgvQPiJWVyv7JfByRFQf67wxRsStEVESESWN27QqcBpmZmYNhx/Kvf0QcGdEXLpZYXJtXz4VpH8QSGoE7FKgj7HA8RFRJukM4EtbG2zqo/RnJbk/a+uA5jnKq9JXtuZp/bwiYpykacDXgL9L+iF5xg0gvQZy73SzFVA9KQSokNQos9Qs6UvA14EBOeoWjNHMzGx745nF7ceTwEhJuwNIai9pX2AaMFjSbuny6ElZxywmmVWD5LrDpun7J4AzMzeMSGqflrcmuUmjKZtfe7c63VfdAqBYUrd0+9vA1E9xTq8A3QrWSnQnXbJO70r+TfUKkvYHXo+IP5JcZ3gg+ccNkmXoe0muPbwtT78LSGciJbUD7gBOzzHbuFmMZmZmOwoni9uJiJgHXAZMkTSHJOHrHBFvA1eSLME+R5KAZdxGkkiWAYcAH6ZtPUZyHWJpujSbWYL9OUny+RwwP6udCcDF6Y0sm77DKyLWA2cCf0mXrquAWz7FOc0H2qY3uhRyJMld0QBdgQ9y1DkZKE/PqQ9wV75xkzQYGARcHRH3AhsknZmjzUf4eIb1LGB34M/VHp9zSo4YzczMdghKbjQ1qx+SLgBWp9cP5qvTCRgXEUel2/cAF6SPq9nW8XUmSTq/UoO6zwAjImLlluo179oliq/+SW2FaDXg74aue/5uaKstme+GljQjIkrqO56dkWcWrb79mY+vbcynC8kdywBExLfqIlFM+3obuE1Smy3Vk9QRuL5QomhmZra98cyiWR0rKSmJ0tLS+g7DzGy74pnF+uOZRTMzMzPLy8mimZmZmeXlZNHMzMzM8nKyaGZmZmZ5OVk0MzMzs7ycLJqZmZlZXk4WzczMzCwvJ4tmZmZmlpeTRTMzMzPLy8mimZmZmeXVpL4DMDMzM9saM2bM2L1JkyZjgD54AmxrVQHlFRUV/zVw4MB3c1VwsmhmZmbbpSZNmozZY489PtexY8eVjRo1ivqOZ3tUVVWl5cuX91q2bNkYYHiuOs7CzczMbHvVp2PHjh84Udx6jRo1io4dO64imZ3NyTOLZnWsfOW79Jx4U531N3/iL2u1vfJXu9Zqe2ZmhVxR1SPfrkZOFD+7dAzzTiB6ZtHMzMzM8nKyaGZmZmZ5OVncjkk6Q1KtrmdKOl5Sr6ztqyQNqaW2Py/pfyX1lvSqpKKsfY9IOlXSsZKuqo3+akMm5vT9HZJmZ70WS3on3XeepO/Wb7RmZma1z9csWnXHA38D5gFExOW12Pb/AP8vIl6W9CDwM+AySccDTSNivCQBv5T024hYW5NGJTWJiIpajPMTMQNExJlZfTYCngbuSotuB55Lf5qZWT3oOfGmgbXZ3vyR580oVGfJkiVNzz333H0WL17cbM2aNY07deq0Ydq0aa9ee+21HWbNmtVy3LhxS2ozpvrgmcUGStK3JE1PZ7BGS2qclp+ZzspNBw7Lqj9W0sis7TVZ738qaa6kMkm/Tcu+L+mltOwBSS0kHUpy2/y1ab9ds9uVdJSkWWlbt0tqlpYvlvQLSTPTfT1znE9r4MCIKEuLrgJOktQf+C1wLkBEBEkSdmyB8TlD0mRJTwFPSmol6cmsGEZk1T1d0pz0XO9Oyzqm5/1S+josRx/VY872P8DyiBiTxr0WWCzpoC3FbWZmO5ZTTz11vxEjRqwsLy9/ZfHixeU33njjvwDmzp3bom/fvjWa9GjonCw2QJI+B5wCHBYR/YFK4DRJnYFfkCSJhwO98reyqa1jgBHAwRHRD7gm3fVgRAxKy14BvhcRzwOTgYsjon9EvJbVTnNgLHBKRPQlmZU+O6urFRExAPgzcFGOUEqA8sxGmlxdBDwDTIiIhVl1S4EvFjo3YAAwMiIGA+uBE9IYjgR+p0Rv4DLgy+m5/nd67A3A7yNiEPB1YEyhmDPShPC/gO9X25U3bkk/kFQqqbTygzW5qpiZ2XamoqKC6dOnt/7qV7+6OlN20EEHrQOYN29eUf/+/dfVX3S1x8liw3QUMBB4SdLsdHt/4GDg6YhYHhEbgPtq0NYQ4I7Mkm5EvJ+W95H0rKS5wGlA7wLt9ADeiIhX0+07gSOy9j+Y/pwBFOc4vjOwPLsgIh4G/gPcXK3uu8CeBeIBeCLrfAT8WtIc4P+AvYBOwJeBv0TEirTPTP0hwE3p+E4G2khqVSjmtM49JMn1+9Xq5407Im6NiJKIKGncpno3Zma2PWrSpAmHHHLIB5///Od7f/Ob39x3ypQpLTP7Fi5cWDRo0CAni7bNCLgznd3rHxE9IuLKAsdUkP4+0+vpdilQfyxwXjpL+Aug+WcLmY/Sn5XkvhZ2XZ4+qtJXtuZp/UI+zHp/GtARGJjOxr6Tp7+MRsAXssZ4r4ioPuWXK+Ybgb9GxJM52qxp3GZmtoOYOnXqwnHjxr3Wpk2byhNOOKH73XffveuiRYuaFhUVVXXo0KGyvuOrDU4WG6YngZGSdgeQ1F7SvsA0YLCk3SQ1BU7KOmYxyWwkJNcdNk3fPwGcKalFpq20vDXwdtrOaVntrE73VbcAKJbULd3+NjD1U5zTK0C3grUS3UmXfyWdIOk3NTimLfBuRGyUdCSwb1r+FMm1kbul7WXOfwrwo8zB6bWTW4w5vXazH8mNOVuM28zMdg6NGjXiq1/96ppbbrnlzeOOO+79srKyohkzZrTo2bPnDnG9IjhZbJAiYh7JdXZT0mXVJ4DOEfE2cCXwAsmdt69kHXYbSSJZBhxCOusWEY+RLLOWpkuumesJf06SfD4HzM9qZwJwcXojy6av6oiI9cCZwF/Spesq4JZPcU7zgbbpTSOFHAk8kr7vCnxQg2PuBUrS2E4nPaeIeBn4FTA1HZvr0/qj0vpzJM0DzqpBzL8imb3M3HiUeWUeAXQYye/KzMx2Ag888ECb9evXC+Ctt95qMn369FZHH330B2VlZUW9evXaYVaalNx8arbtSboAWJ25gzhPnU7AuIg4Kt2+B7ggIpbnO2ZbqknMab3PAz+OiG8XarN51y5RfPVPaivEgvx1f2a2vbuiqgcPlo2dEREl2eVlZWWL+/Xrt6K+4vrGN76x73PPPde6RYsWVbvsskv8+Mc/XnbmmWeuHD58+H7Dhg1bddZZZ1W/tr3BKisr69CvX7/iXPv8nEWrS39m86XzXLoAF2Y2IuJb2zSiwmoSM0AHktlaMzPbSUyYMCHnMxQnT578Rl3Hsi05WbQ6ky5l312gzkt1FE6N1CTmtF6Nl5/7tNud0pHnfaa4PpVa7qtPrbZmZlbYA4A0tr7D2Gn5mkUzMzMzy8vJopmZmZnl5WTRzMzMzPJysmhmZmZmeTlZNDMzM7O8nCyamZmZWV5OFs3MzMwsLz9n0czMzHYM3+g0sFbbm/DOjJpUu/3229v94Q9/6LRhw4ZGFRUVGjFixPt77LHHxlmzZrUcN25czgd3b088s2hmZma2lW688cbdrr/++j0eeuih1+bPnz9v1qxZr7Rv375y7ty5Lfr27bu2vuOrDU4WzczMzLbC+++/3+jnP//5Pvfff/9rXbt23QjQunXrqp/97Gfvzps3r6h///7r6jvG2uBk0czMzGwrjBs3rl2/fv3W9OrVa0P1fQsXLiwaNGiQk0UzMzOznVV5eXlR3759P5EQLlq0qGlRUVFVhw4dKusjrtrmZNHMzMxsK7Rs2bKyqqrqE+UzZsxo0bNnzx3iekXw3dBmda585bv0nHhTnfU3f+Iva7W9r88fVqvtmZkV8sDsO+o7hJyOPfbYVaeffvr+l1566Tv77LNPxbp163TzzTfv9t577zXp1avXDrEEDU4WzczMbEdRw0fd1JYjjzxy7SWXXPLvr3zlK92rqqqoqKjQSSed9N4rr7xSNGzYsFV1Gcu25GTRzMzMbCude+6575977rnv13cc25KvWdxJSCqSNFVSS0nzJfXN2nexpNGSOkp6rD7j3BqS+ksalrU9XNIltdR2ZtwaSzpX0uysV7mkkPQ5SX0lja2NPs3MzBoSJ4sNiBLb6nfyXeDBiPgQOB+4Oe1vL+As4JKIWA68LemwbRTDttIf2JQsRsTkiPhtLbWdGbfKiPhTRPTPvIDJwL0R8UpEzAX2ltSllvo1MzNrEJws1jNJxZIWSLoLKAf2kfRnSaWSXpb0i6y6gyQ9L6lM0nRJrdMZr2slvSRpjqQf5unqNOCvABHxGPA2cDrwe+DKiFiZ1puU1i0U94/TmbVySednlZ+exlEm6e60rJOkh9KyMkmHpuddnnXcRZKuTN8/LemGrNm7g9LygyS9IGlWOg49JO0CXAWcktY/RdIZkm7KGt+n0piezCRzksZK+mPazuuSRhYat2rnfwRwMnBOVvHDwDcKjZ2Zmdn2xNcsNgwHAN+JiBcBJP0sIt6X1Bh4UtKBwHzgPuCUiHhJUhtgHfA9YFVEDJLUDHhO0pSIeCPTeJpQ7R8Ri7P6PB+YDiyMiLuzykuB/7elYCUNBM4EDgYETJM0FdgAXAYcGhErJLVPD/kjMDUiTkjPqRXQrsCYtIiI/mlSdjvQJx2DL0ZEhaQhwK8j4uuSLgdKIuK8NL4zstq5EbgzIu6U9N00luPTfZ2Bw4GeJLOEE6udZ65xQ9KuwFjg2xHxQdauUuAS4JoC52ZmZrbdcLLYMCzJJIqpkyX9gOT30xnoBQTwdkS8BJBJUiQNBQ7MmhlrS5J8vpHVXgfgP9kdRsS/JT0F/K1aLO8CexaI93DgoXRJG0kPAl9MY/xLRKxI+8hc8PtlkllMIqISWCWpULI4Pq3/jKQ2aYLWGrhT0gFpX00LtAFwCHBi+v5uNk/kJkVEFTBPUqccx35i3FK3AHdHxHPVyvOOXfr7/AFAkw6FTt3MzKzhcLLYMHyYeSNpP+AiYFBErExvmmi+hWMF/CgiHt9CnXV52qhKX9map/W3tQo2vwyienyRY/uXwD/SGcpi4OnPGMNHWe+VY/8nxk3Sd4B9gW/lqJ937CLiVuBWgOZdu1Q/NzMzswbL1yw2PG1IksdV6WzXMWn5AqCzpEEA6fWKTYDHgbMlNU3Lu0tqmd1gej1iY0lbSjozupNcO4mkvSQ9maPOs8DxklqkfZ2Qlj0FnCRpt/T4zDL0k8DZaVljSW2Bd4DdJe2WLp8fW62PU9L6h5Mss68imTV9K91/Rlbd1SSzjrk8z8fXEZ6Wxlkj1cdN0v7Ar4HTIqIixyGbxs7MzGxH4WSxgYmIMmAWyfV544Dn0vINJAnUjZLKgCdIZrLGAPOAmekNI6PJPWM8hWT5uJAjgUfS951JZgCrxziT5Jq96cA0YExEzIqIl4FfAVPTGK9PD/lv4EhJc4EZQK+I2EhyY8r09FzmV+tmvaRZJEu+30vLrgF+k5Znn+M/gF6ZG1yqtfMj4ExJc4Bvp7F8Gtnj9lOgBfCgNn+EzhfT/dljZ2ZmtkNQhFfEdgaSBgAXRMS3C9R7BhiRLoGfByyNiMl1EuTHMTwNXBQRpXXZb55YajpuzYCpwOF5Zh03ad61SxRf/ZNajHLL/HV/Zra9e2D2HUiaEREl2eVlZWWL+/XrtyKzXT7g0IG12W+fmc8X/EaYs846a+/dd99941VXXfUOQFVVFW3btu3/1ltvzRk9enT7WbNmtRw3btySNWvW6Mgjj+z+wgsvLGjSpAnXXHNNx7KysqJ77713KcCoUaP2XLp0abNJkya9Ub2PmTNnNj/uuOMOWLp06dzGjRtTUVHB4MGDD1iyZEnz1157rbxZs2YBMHLkyOLy8vIWmeNWrFjRdOPGjXrnnXfKDj/88O4vvPDCgqZNc1/uX1ZW1qFfv37FufZ5ZnEnkc4G/iO9GzknSR2B6zOP0YmIm+o6UWxoajJuqS4kz6rcYqJoZmY7lnnz5hX1799/0/Xq8+fP36V9+/YVbdq0qZo7d26Lvn37rgW48cYbOwwfPnxlkybJwtjZZ5/93pNPPrnrihUrGo8fP77tE088ses999yzOFcfAwYMWN+1a9f1EyZMaAswatSovbp167b+y1/+8qonnniiVabexIkTF8+fP3/e/Pnz502aNGlR48aN4+abb36jefPmMXjw4A/GjBnTPlf7hThZ3IlExO3p3cj59i+PiEl1GVOeOL7UEGYVMwqNW1pnYUQ8XUchmZlZA7FgwYKikpKStZntGTNmtOjZs+c62DyRvP/++3c7+eSTNz1ho3Xr1lXHH3/8+xdccMFeF198cZeJEye+1qpVqwAYPHhwt8WLF282BXj++ee/M3r06N3Hjh2767Rp01rdeuutbw4bNmzV3//+9zbVY3r77bebHH300d0vuuiif5922mmrAEaOHPmfCRMmbFWy6LuhzepYn3a7UzryvLrrsJb7eqBWWzMz234tX768cUVFhbp06bJpVamsrKyoT58+awEWLlxYNGjQoHXr16/Xv/71r2Y9evTYkH38D3/4wxUDBgzofc899yzq3bv3pid0TJ06dVH1vk488cQPLrnkkr2vvPLKvZ955pn5zZo1i6997Wurf/3rX3fOrrd69epGxxxzTLcRI0a8f/HFF29aoh80aNC6OXPmtKzebk14ZtHMzMxsK5SWlhb16NFjs0emlZeXF/Xr12/dokWLmhYVFVV16NChctmyZU1at279icuULr/88s7t2rWrqKioyPX4tk8oKSn58Jxzznknk5y2bt26qk2bNpWZWciKigpGjBixf7du3dbfcMMN/84+tkmTJjRt2jRWrlz5qXM/J4tmZmZmW2HevHnNi4uLN80IVlZWMmPGjFZDhw5dnS5HrwVo2bJl1YYNGzbLua644opOH330UaO77rrr9ZtuuinXF0N8woIFC5oPHDhwbXbZkCFDPpg0aVIbgNNPP33fiooKjR8/fnGu4zdu3KgWLVp86jubnSyamZmZbYX99ttvw+zZs1usXbtWAJdeemnngw46aM0ee+xRWVZWVtSrV691AB07dqysrKxUpt7kyZNbjxs3rsN99933xrHHHrt6zZo1jZ9//vmiTLuHHHJI9zfeeOMTty0vWrSoqKSkZLOZzOHDh6+aMmVK2wsvvLDznDlzWjz88MOv5brjedmyZY133XXXisyd05+Gr1k0MzOzHUJNHnVTm0488cQPHn300TYHHHBAn6ZNm8aAAQPWjB07dgkky9HDhg1blal7xBFHrJoyZUqr3r17f3TOOecUT548eWG7du2qAM4+++x3fve733U69NBDF1dWVrJkyZJmHTt23GzZetGiRU1bt25d2bZt282+eW3AgAHrn3322TaPPvpouz333HPDwQcf3DOzr2vXrusfeeSR1wEeffTRNkOGDFnFVvBzFs3qWElJSZSWNpibvc3Mtgs1ec5iQ/bPf/6zxXXXXdcp13MUs7300kvNR48e3WHMmDFv1rTtb37zm13OOOOM94YOHfphvjpDhw7tet1117154IEHfpRr/5aes+iZRTMzM7Nt7PDDD19bWlr6QUVFBZlnLeYyaNCg9YMGDapxoggwbty4pVvav379eg0fPvw/+RLFQpwsmpmZmdWB888//7366Ld58+Zx3nnnbXXfvsHFzMzMzPJysmhmZmZmeTlZNDMzM7O8nCyamZmZWV5OFs3MzMwsL98NbVbHyle+S8+JN9V3GPYpzJ/4y/oOYadS/mrX+g7BGpg+M5+v7xB2ap5ZNDMzM7O8PLNoZmZmO4Sv9z9zYG2298DsOwp+feBZZ5219+67777xqquuegegqqqKtm3b9n/rrbfmjB49uv2sWbNajhs3bsmaNWt05JFHdn/hhRcWNGnShGuuuaZjWVlZ0b333rsUYNSoUXsuXbq0Wb5veBk5cmRxeXl5i8z2ihUrmm7cuFF77bXXR7NmzZrfrFmzmDlzZvPjjjvugKVLl85t3LgxFRUVDB48+IBTTz31/bFjx3Z44YUXFuT63uhCPLNoZmZmtpXmzZtX1L9//3WZ7fnz5+/Svn37ijZt2lTNnTu3Rd++fdcC3HjjjR2GDx++MvPtLWefffZ7Tz755K4rVqxoPH78+LZPPPHErvfcc8/ifP1MnDhx8fz58+fNnz9/3qRJkxY1btw4br755jdKSko+fOKJJ1pB8j3RXbt2XT9hwoS2AKNGjdqrW7du688777z3Bg8e/MGYMWPab805Olm0eiXpfEmnS/q+pPuyyttIek3S/pKuk/Tleoqvs6S/pe9/JWl21utVSZWSWkk6VtJV9RGjmZnVnwULFhSVlJSszWzPmDGjRc+ePdfB5onk/fffv9vJJ5/8n0y91q1bVx1//PHvX3DBBXtdfPHFXSZOnPhaq1atAmDw4MHdFi9enHMK8O23325y9NFHd7/ooov+fdppp60aNmzYqr///e9tMvvPP//8d0aPHr372LFjd502bVqrW2+99U2AkSNH/mfChAlOFq32SdpmlyqkbX8XGAeMAfaRNCTdfRVwe0S8DtwIXLKt4ijgx8BtABHxs4jon3kBLwG/iYg1wCPAcZJabKEtMzPbgSxfvrxxRUWFunTpUpEpKysrK+rTp89agIULFxYNGjRo3fr16/Wvf/2rWY8ePTZkH//DH/5wxT333NPx6quvXtq7d+9N39s8derURcXFxRur97d69epGxxxzTLcRI0a8f/HFF68A+NrXvrb6n//8Z+tMnRNPPPGDZcuWNb3yyiv3fuihh15r1qxZAAwaNGjdnDlzWm7NeTpZ3A5ImiRphqSXJf0gq/x76ezWdEm3SbopLe8o6QFJL6Wvw/K0O0zS/LTtP2bNoF0p6W5JzwF3S2ou6Q5JcyXNknRkWu+MTJ/p9t8kfSl9v0bS79OYn5TUMUcIXwZmRkRFRARwFvAHSSXAUcC1ABGxBNhN0h4FxukkSeWSyiQ9k5Y1lnRtOg5zJP0wLT8hjUvp7OGredr/OvBYjr6+BXQDrkxjDOBp4NgtxWhmZjuO0tLSoh49eqzLLisvLy/q16/fukWLFjUtKiqq6tChQ+WyZcuatG7duqL68Zdffnnndu3aVVRUVKhQXxUVFYwYMWL/bt26rb/hhhv+nSlv3bp1VZs2bSqzZyJLSko+POecc97JTmKbNGlC06ZNY+XKlZ8693OyuH34bkQMBEqAUZJ2k7Qn8HPgC8BhQM+s+jcAv4+IQSTJzpjqDUpqDowGjknbrp7M9QKGRMSpwLkk+VBf4FTgzvT4LWkJlEZEb2AqcEWOOocBmy4ejog5wOPAk8CPIiL7L7CZaf0tuRz4akT0A4anZd8DVqVjMQj4vqT9IuIh4O303G4DroiIZdmNSdoPWBkRH1UrLwZ+C5wWEdn/+EuBL+YKTNIPJJVKKq38YE2B0zAzs+3BvHnzmhcXF2/6f0RlZSUzZsxoNXTo0NXpcvRagJYtW1Zt2LBhs5zriiuu6PTRRx81uuuuu16/6aabOhXq6/TTT9+3oqJC48ePX1x935AhQz6YNGnSpqXoBQsWNB84cODa6vU2btyoFi1axKc7SyeL24tRksqAF4F9gAOAg4CpEfF+RGwE/pJVfwhwk6TZwGSgjaRW1drsCbweEZm7rsZX2z85IjJ/LR0O3AMQEfOBJUD3AjFXAZlrEO9J26iuM7C8WtmfgLci4ulq5e8Cexbo8zlgrKTvA43TsqHA6elYTAN2Ixk/gB8BlwIfRUT1888Zn6TG6fn8PCIW1TTGiLg1IkoioqRxm+q/CjMz2x7tt99+G2bPnt1i7dq1AkunQq0AABG4SURBVLj00ks7H3TQQWv22GOPyrKysqJevXqtA+jYsWNlZWWlMvUmT57cety4cR3uu+++N4499tjVa9asafz8888XZdo95JBDur/xxhubZgovvPDCznPmzGnx8MMPv5brbubhw4evmjJlStvM9qJFi4pKSko2m/FctmxZ41133bUisyz9afjROQ1cuqw7BDgkItZKehooNKvXCPhCRKyv1tbjQCeSGbBCT4X+sAbhVbD5HxxbiivXh3NdjmOq0ld1zdP6+TuIOEvSwcDXgBmSBgIimaV8PMche6d9dZLUKCKq95srvsuAtyPijq2J0czMtp2aPOqmNp144okfPProo20OOOCAPk2bNo0BAwasGTt27BJIlqOHDRu2KlP3iCOOWDVlypRWvXv3/uicc84pnjx58sJ27dpVAZx99tnv/O53v+t06KGHLq6srGTJkiXNOnbsWAGwYMGCXa6//vo999xzzw0HH3zwplXErl27rn/kkUdeh+Qu6Ndff715RUUFixcvbtq6devKtm3bbvb/tEcffbTNkCFDVrEVnCw2fG1JlkLXSupJsuwMyc0Vf5DUDlhNstw8N903hWTW7FoASf0jYnZEfDXTqKQiYH9JxRGxGDhlCzE8C5wGPCWpO9AFWAC0Ac6R1AjYi2S2M6MRMBKYAHwT+GeOdl8hue6vJrqTzp5K+g0wPV1K3kRS14iYBkyTdAzJLOzjwNmSnoqIjWn8bwEfAbeTLKt/h+RGluuq9fkqUJzV/heAM4ABW4ixvIbnY2ZmO4DbbrvtTeDN6uWTJ0/e7HmJo0aNWn7dddd1mjRp0htvvvnm3Gr73hs1atR7ADNnzmw+bNiwlZk7o3v06LEhIgomwQMGDFjz1FNPtRw6dOiH1dsHGD9+fPvrrrvuE3HWhJPFhu8x4CxJr5AkaC8CRMRbkn4NTAfeB+YDmb8YRgF/kjSH5Hf8DMnNI5tExDpJ5wCPSfqQJPnM52bgz5LmkswmnhERH6U3wLwBzCNJ/GZmHfMhcJCky0iWZ3Mlo48CdxcaAElNSZLK0rSoL8nyenXXSjqAZDbxSaAMmEOS8M2UJJJl5eOBC4FnI+Kf6RL/S5IeiYhXMo1FxIfp43u6pUvOvwBaAP9Imtrk6xHxGnAkybK2mZnZZg4//PC1paWlH1RUVJB51mIugwYNWj9o0KBPndSNGzduab5969ev1/Dhw/9z4IEHfpSvzpYouYnTtkeSWkXEmvQRNA+RPGrmoULH5TheJNcKLoyI39dSbGsiouDFeZIeAn4SEQu3UOcEYEBE/Dzdfjx7lnRbSvseGBGXFajXCRgXEUcVarN51y5RfPVPaitEqwP+bui65e+Gtur6zHweSTMioiS7vKysbHG/fv1W1FdcO5KysrIO/fr1K861zze4bN+uTG/cKCeZ4Zv0KY//fnr8yyTL3aNrOb6auITkRpItaQL8LrNRV4li2tdDwOIaVO1CMltpZma2Q/HMolkdKykpidLS0sIVzcxskzwzi6/37dt3ZaNGjZzMfAZVVVWaO3duu379+u2fa79nFs3MzGx7Vb58+fK2VVVVBR9qbblVVVVp+fLlbdnCDZq+wcXMzMy2SxUVFf+1bNmyMcuWLeuDJ8C2VhVQXlFR8V/5KjhZNDMzs+3SwIED3+Xjb+yybcRZuJmZmZnl5WTRzMzMzPJysmhmZmZmeTlZNDMzM7O8nCyamZmZWV5OFs3MzMwsLyeLZmZmZpaXk0UzMzMzy8vJopmZmZnl5WTRzMzMzPLy1/2Z1bHyle/Sc+JN9R2G1bL5E39Z3yHUq6/PH1bfIdgO7IHZd9R3CDs1zyyamZmZWV5OFs3MzMwsry0mi5KKJZXXVTDbI0mLJXWo7zgKkdRZ0t8k7Z7GvEfWvj9JulRSX0lj6zHMrSLpS5IOzdo+S9LptdR2Z0l/S9//StLsrNerkioltZJ0rKSraqNPMzOzhqReZhYl1cm1kpIa10U/tUGJbfn7+DFwW0S8C/wWuC7tdwDwReC6iJgL7C2pyzaMY1v4ErApWYyIWyLirlpq+8fAbWm7P4uI/pkX8BLwm4hYAzwCHCepRS31a2Zm1iDUJDlpLOk2SS9LmiKpCEBSf0kvSpoj6SFJ7dLypyWVpO87SFqcvj9D0mRJTwFPpjM2z6QzNOWSvli94/SYv6ZtLpR0Rda+b0manh4/OpMYSloj6XeSyoBDqrX3J0nD0/cPSbo9ff9dSb8q0O5QSS9IminpL5JaVWu7SNKjkr6f4zw6SnoiHcMxkpakY1MsaYGku4ByYB9Jp0qam47J1VltrMl6PzIzAyhprKRbJJWmM13H5vk9fh14LH1/K9BV0pHAn4DzImJjuu9h4Bt52sj0L0nXpjHOlXRK1r6fpmVlkn6blnWT9H9p2UxJXdPZwL9lHXeTpDPS94slXZO2M11St7T8OEnTJM1K2+skqRg4C7gg/Z19UdKVki5Kj9nS5/TqtP1Xc33+coxb9hh8C+gGXAkQEQE8DeQbfzMzs+1STZLFA4A/RURv4D8k//MEuAv4aUQcCMwFrshzfLYBwMiIGAx8E3g8naHpB8zOc8xBaZ8HAidJKpH0OeAU4LD0+ErgtLR+S2BaRPSLiH9Wa+tZklk0gL2AXun7LwLP5GtXyTLzZcCQiBgAlJLMOGW0IkmyxkfEbTnO4QrgqXQMJwLZM3cHADen+zYCVwNfBvoDgyQdn2dcshWn4/Q14BZJzbN3StoPWBkRHwFERBVwNvAAsCAinsmqXsrHY5TPiWl8/YAhwLVp8n8MMAI4OCL6Adek9e8l+Qz1I5kBfLsG57QqIvoCNwF/SMv+CXwhIj4PTAB+EhGLgVuA36czfs9Wa2dLn9MmEXEQcD45Pr/Vxy2rvJhkdva0iKjI2lWTsTMzM9uu1GQ5+I2IyCRyM4BiSW2BXSNialp+J/CXGrT1RES8n75/CbhdUlNgUlYfuY55D0DSg8DhQAUwEHhJEkAR8G5av5IkCcrlWeB8Sb2AeUA7SZ1JZiBHAd/J0+4XSBLL59LyXYAXstr9K3BNRNybp9/DgRMAIuIxSSuz9i2JiBfT94OApyNieXq+9wJHAJPytJtxf5oALpT0OtCTzZPvzsDy7AMiYraS61FvrtbWu8CeBfo7nCQxrgTekTQ1jX0wcEdErE37eF9Sa2CviHgoLVufnluBLhif9fP36fu9gfvS39kuwBtbaqAGn9MH058zSBLu6j4xbulM8z3AzyNiUbX6ecdO0g+AHwA06dBuS2GbmZk1KDWZWcyeVamkcIJZkdVu82r7Psy8SWezjgDeAsZKOl3SCfr45oGSTNVqbQQg4M6s68d6RMSV6f71aRKDpIOz2hseEW8BuwJHA8+QJI8nA2siYvUW2hVJ0pop7xUR38uK6TngaKUZkKRzs/otlHh9WGB/9nlnVB/XXGOUbV2OYwCq0le25mn9bS37c5LpN1vkeH8jcFM64/jDHMd8WpnPdr7Pda5xuwx4OyJyPfQr79hFxK0RURIRJY3btMpVxczMrEHaqhsqImIVsDLrOq9vA5nZm8Uks3MAI/O1IWlf4J102XYMMCAiHspKyErTql+R1F7JtZLHkyRmTwIjJe2ettU+ba96nNOy2pucFr9IsuyYSRYvSn+yhXZfBA7LunaupaTuWV1dDqwkuf6PiPhTVr//TmM+OT12KJBvamk6MDi9nrExcGrWuL4j6XNKboI5odpxJ0lqJKkrsD+woNr+V8k9c5ZLd5LrJ5F0UHo9ZXXPAqdIaiypI0nSPx14AjhT6U0ektqnSfibmeV0Sc3S/UuAXun2rsBR1fo4JetnZha3LckfF5DMAmesBlpXD7LA57QmNhs3SV8AziCdIcxh09iZmZntKD7L3bffIblWbQ7J9WuZx4ZcB5wtaRawpUfKfAkoS+udAtyQp950kmXlOcADEVEaEfNIZnimpP0/QbJkWBPPklyrtgiYCbRPy8jXbrosfAYwPi1/gWSpN9t/A0WSruGTfgEMTZd9TwKWkSQ4m4mIt4FLgH8AZcCMiPhruvsS4G/A83zymr+lJOP0KHBWZqk3q90PgdcyyW4BR5Lc2QvJtZW5ZsoeIvl9lAFPkVw7uCwiHgMmA6WSZpMk4pAkaaPSsXse2CMi/gXcT5Jc3Q/MqtZHu7T+fwMXpGVXAn+RNANYkVX3YSAzK139msF8n9OCcozbL4AWwD+0+SN0uqb7s8fOzMxsh6DkJs6GKb07tiQizqvvWD4LSc2AyoiokHQI8Of0BpraaHss8LeImFig3gnAwIi4rECcU4HD01ivBe6OiDm1EWtNKbmDviQiVhSqWwexFBy3tF4nYFxEVJ8h/YTmXbtE8dU/qa0QrYHw1/356/5s23lg9h1ImhERJYVrW23zd0PXjS7A/ekS8gbgE4/X2dYi4iFJuxWo1gW4JHOHb0RcvO0ja9hqOG6QjN2F2zoeMzOzutagk8WIGAuMrecwPrOIWAh8fhu1fcanqDumwP6FwMLPGtNnFRHF9R1DtkLjltZ5qS5iMTMzq2sNehnabEdUUlISpaWlhSuamdkmXoauP/XydX9mZmZmtn1wsmhmZmZmeTlZNDMzM7O8nCyamZmZWV5OFs3MzMwsL98NbVbHJK3mk1/JuLPqwObfxrMz81h8zGPxMY9FogPQMiI61ncgO6MG/ZxFsx3UAj/+ISGp1GOR8Fh8zGPxMY9FIh2H4vqOY2flZWgzMzMzy8vJopmZmZnl5WTRrO7dWt8BNCAei495LD7msfiYxyLhcahHvsHFzMzMzPLyzKKZmZmZ5eVk0WwbkHS0pAWSFkm6JMf+ZpLuS/dPk1Rc91HWjRqMxY8lzZM0R9KTkvatjzjrQqGxyKr3dUkhaYe9C7YmYyHp5PSz8bKkcXUdY12pwb+RLpL+IWlW+u9kWH3EWRck3S7pXUnlefZL0h/TsZojaUBdx7gzcrJoVsskNQb+BBwD9AJOldSrWrXvASsjohvwe+Dquo2ybtRwLGYBJRFxIDARuKZuo6wbNRwLJLUG/huYVrcR1p2ajIWkA4BLgcMiojdwfp0HWgdq+Lm4DLg/Ij4PfAO4uW6jrFNjgaO3sP8Y4ID09QPgz3UQ007PyaJZ7TsIWBQRr0fEBmACMKJanRHAnen7icBRklSHMdaVgmMREf+IiLXp5ovA3nUcY12pyecC4Jckfzysr8vg6lhNxuL7wJ8iYiVARLxbxzHWlZqMRQBt0vdtgX/XYXx1KiKeAd7fQpURwF2ReBHYVVLnuolu5+Vk0az27QX8K2v7zbQsZ52IqABWAbvVSXR1qyZjke17wKPbNKL6U3As0iW1fSLikboMrB7U5HPRHegu6TlJL0ra0mzT9qwmY3El8C1JbwJ/B35UN6E1SJ/2vylWC/wNLmbWIEj6FlACDK7vWOqDpEbA9cAZ9RxKQ9GEZKnxSySzzc9I6hsR/6nXqOrHqcDYiPidpEOAuyX1iYiq+g7Mdg6eWTSrfW8B+2Rt752W5awjqQnJ0tJ7dRJd3arJWCBpCPAzYHhEfFRHsdW1QmPRGugDPC1pMfAFYPIOepNLTT4XbwKTI2JjRLwBvEqSPO5oajIW3wPuB4iIF4DmJN+VvDOq0X9TrHY5WTSrfS8BB0jaT9IuJBekT65WZzLwnfT9SOCp2DEfelpwLCR9HhhNkijuqNelQYGxiIhVEdEhIorT78B9kWRMSusn3G2qJv9GJpHMKiKpA8my9Ot1GWQdqclYLAWOApD0OZJkcXmdRtlwTAZOT++K/gKwKiLeru+gdnRehjarZRFRIek84HGgMXB7RLws6SqgNCImA/9LspS0iORi7m/UX8TbTg3H4lqgFfCX9B6fpRExvN6C3kZqOBY7hRqOxePAUEnzgErg4ojY4WbfazgWFwK3SbqA5GaXM3bQPy6RNJ7kj4QO6TWaVwBNASLiFpJrNocBi4C1wJn1E+nOxd/gYmZmZmZ5eRnazMzMzPJysmhmZmZmeTlZNDMzM7O8nCyamZmZWV5OFs3MzMwsLyeLZmZmZpaXk0UzMzMzy8vJopmZmZnl9f8BvF4ycu/mBg4AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] @@ -1205,11 +1193,11 @@ ], "source": [ "plot([\n", - " decomp_S_EG,\n", - " decomp_S_ER,\n", - " decomp_S_RO,\n", - " decomp_S_AG,\n", - " decomp_S_HO,\n", + " decomp_X_EG,\n", + " decomp_X_ER,\n", + " decomp_X_RO,\n", + " decomp_X_AG,\n", + " decomp_X_HO,\n", "][::-1])" ] }, @@ -1217,9 +1205,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "When looking at `education` and `sex`, we can see that `education` uniquely contains considerable amount of information about `income` with respect to `sex`. The proportion is also large than what they convey about `income` in both shared and complementary aspects. Notably, `education` have large unique information about `income` when considering with `race`, and `occupation` also has a similar relative portion with respect to `race`.\n", + "We see that most of the information that race ($Y$) and occupation ($Z$) convey about income ($X$), is uniquely in the occupation. A more classical approach would be, of course, to test for the Markov relation $X-Z-Y$. This Markov relation almost holds, in the sense that $I(X;Y|Z)$ is small. The additional insight, due to the information decomposition, is that $I(X;Y|Z)$ is purely synergistic since there is no unique information that race conveys about income w.r.t. occupation. The phenomenon is even stronger for the pair (age, sex): The conditional mutual information is (relatively) larger, but still there is no unique information that sex conveys about income w.r.t. age. Education and sex have about equally large shared and synergistic components. Occupation conveys a large unique information about income both w.r.t. hours-per-week and education.\n", "\n", - "This way of decomposition provides us a new aspect for analyzing the interactions between variables (predictors and responses) at a granular level." + "These observations appear quite reasonable. They illustrate how the decomposition allows us to obtain a fine-grained quantitative analysis of the relationships between the predictors and the target. " ] } ], @@ -1255,7 +1243,7 @@ "width": "212px" }, "toc_section_display": "block", - "toc_window_display": true + "toc_window_display": false } }, "nbformat": 4,