diff --git a/examples/Frank's Tutorial on Reweighing.ipynb b/examples/Frank's Tutorial on Reweighing.ipynb new file mode 100644 index 00000000..e4ddf061 --- /dev/null +++ b/examples/Frank's Tutorial on Reweighing.ipynb @@ -0,0 +1,736 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Frank's Indepth Tutorial of the Reweighing Technique\n", + "##### Author: Guanzhong Chen\n", + "##### Date: 04/17/2020\n", + "*Reference: F. Kamiran and T. Calders, \"Data Preprocessing Techniques for Classification without Discrimination,\" Knowledge and Information Systems, 2012.*" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 1. Introduction to Bias in Machine Learning" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This tutorial is meant to introduce one of the techniques in AI Fairness 360 (AIF360) package called \"Reweighing Technique.\" The AIF360 toolkit is an open-source library the helps machine learning researchers and the whole community detect and mitigate bias in machine learning models." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To give a brief background introduction of bias in machine learning, we look at a very simple dataset. This dataset classifies people described by a set of attributes as good or bad credit risks. File used is `german.data` consisting of 1000 instances and 20 features. In this case we focus on the supervised machine learning problem with a binary target of the credit risks being either \"good\" or \"bad.\" A machine learning model will learn and generalize the pattern from a training dataset and make predictions on a test dataset based on what it has learned. However, here is a problem. The training dataset may not be representative of the true population of people of all age groups. For example, in the training dataset, people with ages more than 25 are much more likely to receive a good credit risk due to the source of the dataset or some other reasons. However, the true distribution might be otherwise. This will generate bias and be unfavorable for people with ages less than 25. In this case, \"age\" will be our protected attribute, and it separates the instances into two groups: more than 25 and less than 25." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Before further investigation, let's first import the german dataset, set the protected attribute, set the threshold of separation, set the training and testing dataset, and drop other sensitive attributes." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Matplotlib Error, comment out matplotlib.use('TkAgg')\n" + ] + } + ], + "source": [ + "# Load all necessary packages\n", + "import sys\n", + "sys.path.insert(1, \"../\") \n", + "\n", + "import numpy as np\n", + "np.random.seed(0)\n", + "\n", + "from aif360.datasets import GermanDataset\n", + "from aif360.metrics import BinaryLabelDatasetMetric\n", + "from aif360.algorithms.preprocessing import Reweighing\n", + "\n", + "from IPython.display import Markdown, display" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "dataset_orig = GermanDataset(\n", + " protected_attribute_names=['age'], \n", + " privileged_classes=[lambda x: x >= 25], \n", + " features_to_drop=['personal_status', 'sex'] \n", + ")\n", + "\n", + "dataset_orig_train, dataset_orig_test = dataset_orig.split([0.7], shuffle=True)\n", + "\n", + "privileged_groups = [{'age': 1}]\n", + "unprivileged_groups = [{'age': 0}]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2. Preprocessing Techniques to Mitigate Bias" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "There are many solutions to mitigate bias in machine learning. In this tutorial, we focus on those that are proceeded before training. They are known as the \"preprocessing\" techniques." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We list three of them here that are commonly used. We give a brief introduction to what they are and what their pros and cons are to help readers to choose the one that is most suitable for them." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "1. **Suppresion**: First, we identify the attributes that are most correlated with the protected attribute $A$. Then, we just remove $A$ and these most correlated attribute. \n", + " * **Pros**: the algorithm itself is straightforward to understand and easy to implement. \n", + " * **Cons**: Sometimes we can't get rid of the protected attributes easily by just removing them. Some of them may be critical for companies' bussiness analysis, and some of them may be important for the classification process." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "2. **Dataset Massaging**: We change the labels of some objects in the dataset. Selections of the labels to change will base on a ranker that is related to Naive Bayes classifier.\n", + " * **Pros**: It help mitigate bias even if the protected attributes are not allowed to be removed. It is also relatively easy to understand.\n", + " * **Cons**: It is, in a sense, rather intrusive as it changes the labels of the instances." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "2. **Reweighing**: We give training instances weights according to the law of statistical independence.\n", + " * **Pros**: It is calculation-friendly. We can implement this algorithm using frequency count. Compared to dataset massaging, it also helps mitigate bias without changing the labels.\n", + " * **Cons**: More difficult to implement compared to the two techniques mentioned above." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 3. Elaboration on Reweighing" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### a. Notations and Weight Concept" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To have an idea of how this technique works, we first introduce some notations. We assume the protected attribute and the target variable are binary. Specifically, we denote the protected attribute as $A$ with two values $\\{b,w\\}$. We denote the target class as $T$ with two values $\\{+, -\\}$. The classifier we use is denoted as $C$, and the random unlabeled data subject is denoted as $X$. Also the training dataset is denoted as $D$." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If the dataset $D$ is unbiased, $A$ and $T$ are statistically independent. This means that the expected probability to see an instance with its protected attribute value and class given independence can be calculated as the following: \n", + "\n", + "$$\n", + "P_{exp}(A=b \\wedge T=+) = \\frac{|\\{X \\in D | X(A) = b\\}|}{|D|} * \\frac{|\\{X \\in D | X(T) = +\\}|}{|D|}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In reality, the observed probability to see an instance with its protected attribute value and class is calculated as the following: \n", + "\n", + "$$\n", + "P_{obs}(A=b \\wedge T=+) = \\frac{|\\{X \\in D | X(A) = b \\wedge X(T) = +\\}|}{|D|}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We notice that $P_{exp}$ and $P_{obs}$ are usually different. We assign the weight for each instance to be $W$ that is calculated as the following:\n", + "\n", + "$$\n", + "W(X) = \\frac{P_{exp}(A=b \\wedge T=+)}{P_{obs}(A=b \\wedge T=+)}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We notice that $W(X)$ is essentially the expected probability divided by the observed probability. In this way, we assign lower weights to objects that have been deprived or favored to compensate for the bias." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### b. A Concrete Example of Weight Calculation" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Notations may be abstract for some readers. We therefore look at a concrete example using the same dataset mentioned above." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We recall that our protected attribute here is the \"age.\" And this protected attribute is already binary (1.0 vs 0.0) by setting the threshold 25. If an person is older than or is 25 years old, we label him/her as 1.0 in the age attribute. We label him/her 0.0 otherwise. We also have binary value for the target class. In this case 1.0 means good credit risk, and 2.0 means bad credit risk. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "First we want to calculate the expected probability to see an instance with its protected attribute value and class given independence. Specifically, we look at $P_{exp}(A=1.0 \\wedge T=1.0)$ first." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As the equation indicates above, we need to calculate the frequency count of $A = 1.0$ in the training set first. That is $|\\{X \\in D | X(A) = 1.0\\}|$." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "587" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "freq_count_of_age_bigger_25 = sum(dataset_orig_train.convert_to_dataframe()[0][\"age\"] == 1.0)\n", + "freq_count_of_age_bigger_25" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In other words, in this dataset, **we have a total of 587 people who are older than or are at age 25**." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And the frequency count of T = 1.0 next. That is $|\\{X \\in D | X(T) = 1.0\\}|$." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "490" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "freq_count_of_good_credit = sum(dataset_orig_train.convert_to_dataframe()[0][\"credit\"] == 1.0)\n", + "freq_count_of_good_credit" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In other words, in this dataset, **we have a total of 490 people who have a good credit risk**." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, we need to know the total number of training instance. That is $|D|$." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "700" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "total_instance = len(dataset_orig_train.convert_to_dataframe()[0][\"credit\"])\n", + "total_instance" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Combine the three values above, we have:\n", + "\n", + "$$\n", + "P_{exp}(A=1.0 \\wedge T=1.0) = \\frac{|\\{X \\in D | X(A) = 1.0\\}|}{|D|} * \\frac{|\\{X \\in D | X(T) = 1.0\\}|}{|D|}\n", + "$$\n", + "\n", + "We can calculate this value in Python." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "0.587" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "P_exp = freq_count_of_age_bigger_25/total_instance * freq_count_of_good_credit/total_instance\n", + "P_exp" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next we calculate the observed probability $P_{obs}(A=1.0 \\wedge T=1.0)$." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As the equation indicates above, we need to calculate the frequency count of $A = 1.0$ and $T = 1.0$ in the training set first. That is $|\\{X \\in D | X(A) = 1.0 \\wedge X(T) = 1.0\\}|$." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "427" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df = dataset_orig_train.convert_to_dataframe()[0]\n", + "freq_count_both = df[(df['age']==1.0)&(df['credit']==1.0)].shape[0]\n", + "freq_count_both" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In other words, in this dataset, **we have a total of 427 people who have a good credit risk and older than or are at age 25**." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Since we already the total number of training instance is 700. We can calcuate the $P_{obs}$ easily using the following equation.\n", + "\n", + "$$\n", + "P_{obs}(A=1.0 \\wedge T=1.0) = \\frac{|\\{X \\in D | X(A) = 1.0 \\wedge X(T) = 1.0\\}|}{|D|}\n", + "$$\n", + "\n", + "We can calculate this specific value in Python." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "0.61" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "P_obs = freq_count_both / total_instance\n", + "P_obs" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, we can calculate the weight using the two probabilities.\n", + "\n", + "$$\n", + "W(X) = \\frac{P_{exp}(A=1.0 \\wedge T=1.0)}{P_{obs}(A=1.0 \\wedge T=1.0)}\n", + "$$\n", + "\n", + "Again, the mathematical value can be calculated in Python." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "0.962295081967213" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "weight = P_exp / P_obs\n", + "weight" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "That is the weight we will assign to for each instance with $A = 1.0$ and $T = 1.0$ in the training set." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Summary: this means that for this dataset, we give people who are older than or at age 25 AND have a good credit risk a weight of 0.587.**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Thanks to the AIF360 package, we do not have to go through this weight-assigning preprocessing by hand ourselves. The package has its own function that calculates the weights for us automatically. " + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [], + "source": [ + "RW = Reweighing(unprivileged_groups=unprivileged_groups,\n", + " privileged_groups=privileged_groups)\n", + "dataset_transf_train = RW.fit_transform(dataset_orig_train)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can confirm if our calculation is correct using the package." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.9622950819672131\n" + ] + } + ], + "source": [ + "print(RW.w_p_fav) # S is 1.0 and T is 1.0" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Congragulations! We have achieved the same result." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The other thing to notice is that since $A$ and $T$ are both binary, we have a total of 4 different weights to assign. If reader is interested, he/she can calculate the other 3 values of weight by hand and confirm them with AIF360." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1.100625\n", + "1.2555555555555555\n", + "0.678\n" + ] + } + ], + "source": [ + "print(RW.w_p_unfav)\n", + "print(RW.w_up_fav)\n", + "print(RW.w_up_unfav)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Summary:** \n", + "**1. First value says that for this dataset, we give people who are younger than age 25 AND have a good credit risk a weight of 1.100625.** \n", + "**2. Second value says that for this dataset, we give people who are older than or at age 25 AND have a bad credit risk a weight of 1.2555555555555555.** \n", + "**3. Third value says that for this dataset, we give people who younger than age 25 AND have a bad credit risk a weight of 0.678.**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can further visualize how many instances receive a specific weight value using a bar plot in Python." + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAFdCAYAAADWns55AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dedxVVb3H8c8XUBxzSDAUEA0ckAwT0dLKwimz1HJArTA1GzQzb3rVJr1Fec0Gbw5dM684JGKmkpqmKOKMqIiCEyoFQoJTohkG/u4fax2ezeGcZ8DnPM+G5/t+vc7r7LP2tPb4W3vtfdZWRGBmZmbl0K2zM2BmZmZNHJjNzMxKxIHZzMysRByYzczMSsSB2czMrEQcmM3MzErEgbmTSTpA0mxJb0javoPmebqkyztiXm0l6RJJP26m/xuStmjQvP8saVQjpt3CfH8s6SVJf+/oeTeSpImSju7E+S/dzyX1z/tO93ac/nRJu7UwzABJIalHO873cEl/aab/bpLmtNf82kLSaZIuauWwnXoe6uz5N2eVCcySDpM0JR988/JJdtcOmG9IGvguJnE2cFxErBMRj9SZ/pt5uSqfk9/F/NpMyfGSHs95mSPpakkf6Mh8AOT19Ny7nU6tgzIiPhURY97ttNuYj37AfwCDI+J9zQy3uaR3JJ3fcblrLElb5v3oJUn/kDRN0ontGTwrIuJved9ZkufdbKGhEFArx9wsSadUTXPbiJjY3nltSURcERF7FvK6wucgSYdKmlGVdmudtFNoQUT8JCLapTCW1/nudfptKmmxpPfX6HetpLPbIw+dZZUIzJJOBH4F/ATYGOgPnA/s15n5aqXNgOktDPPBfFKpfM7qiIwVnAN8Czge2BDYErgO+HQH52NVtBnwckTMb2G4LwGvAiMl9Wx8thorn1AfAGYDH4iI9YCDgGHAujWGb7crzjZaPyLWAQ4Evi9pj07KR6PcCWwjqRcsXc8fBNaqSvswMKnTclklIl4AJgBfLKZL2hDYB+jQAna7i4iV+gOsB7wBHNTMMD1JgXtu/vwK6Jn7HQHcXTV8AANz9yXAecCNwELSyeT9ud+kPOybOQ+H1Jh3N+B7wF+B+cClOc898ziV8Z+tk/eleanRbzhwH/AaMA84F1i90H9b4FbgFeBF4LScfjowLudlIalgMKzOPAYBS4DhLWyDS4EFeTm/B3QrrN97gF/mfD4HfCSnz87rZFRhWpcAv8n5Xkg6cWzW1m2T+5+T5/E68BDw0Zy+N/A28O+8DR7N6ROBo5vbbrnfgJyPUcDfgJeA77Z1/QC7A28B7+R8XNLMNJ4Fvp6344FV/fYEngL+QSqQ3llZjtz/SOAJUmC/pbg+a8znauDveVqTgG2rtk1z63sP4Mk87rnV+aiaz+XAjc3ko7KOj8rreFJO3xm4N+9LjwK7FcbZPM9zYd5/zgUur5peD2A0aZ/+V17v5zYz/x6FtMnASYXfs4DdC8filLyvvQj8otZ0gM/n8YbUmOedwOdz9655vH3y792BqdXnLGqcg4DdgDmkmpj5pHPDl1vYtz5fWI47SIGtmPZP8rkF2AS4hrQ/Pw8cX5jW6ZV1nn9/ibTPvwx8v2qdnU6d8xBwGem4eCsv18k18n0YVedN4BvAwy2dA6rzWllnVdMq5rUbcEpeVy/nfG9YZ322ajs291kVrpg/DKwBXNvMMN8lHdBDSaXB4aSTY2sdCpwBbADMJB3YRMTHcv/KFe1VNcY9In8+AWwBrEM6ESyKVBKvjL9clUwrLAG+DWxEWg8jSDsmktYFbgNuJh1IA0klzIrPAmOB9YHxpJNYLSNIO+zkZvLxa1Lw2QL4OOlg/HKh/07ANOC9wO/zfHfMefoCcK6kdQrDHw78KC/XVOCKZuZdc9tkD5K2+YZ5vldLWiMibibVrlyVt9sHa0z3CGpst6phdgW2Iq2jH0japk4ea66fiLgN+BQwN+fjiFojS/oo0Je03sbl8Sv9NgL+AJxKWr9PkQo+lf77A6cBnwN6AXcBV9bJJ8CfSYWx3sDDLL/ua67vnI9rSMfVRqQT2C7NzGf3nO+WfBzYBthL0qakQsGPSdv0O8A1lSs70jZ+KM//R6SC03Ii4ruk9VC5hXRcS5mQtDMwhLTMtZwDnBMR7wHeT9pO1dP4MvDfpJP94zWmcScpQAB8jFSI/Xjh9501lqXeOeh9pH1uU1Lh5jxJG9TJ+6Q8/cp87gLurkq7PyLeltQN+BOpULQpad8/QdJeNZZ3MKmgeDjQp5CfoprnoYj4IqlA9pmoX0t4LbBR1S3LL5ICfUXNc0Cd9dCc44H9SdtjE1Ih97w6w7Z5Oy6npchd9g9po/+9hWGeJZdY8u+9gFlRVfos9K++Kruo0G8f4Mlaw9aZ9wTgG4XfW5Gu1Hq0cvwglfZeK3z2qjPsCcC1uftQ4JE6w50O3Fb4PRh4q86w3yUdlPXy1x1YRLpHWkn7KjCxsH6fKfT7QF6mjQtpLwNDC+t7bKHfOqQCSL+2bpsaeX2VdAKrrIPLq/pPpOmKue52o+kqqG+h/2Rg5Aqsn92oKqnXmMZFwHW5+8M5H73z7y8B9xWGFekKobIcfwaOKvTvRrr62awVx9b6eTkrNQV113fOx/1V+ZhD/SvmfwN7NzPvyjreopD2n8BlVcPdQgrA/YHFwNqFfr+nxhVz9bZuYf6vka7agvQ8iArDzKLpimoSqcCyUZ3pfAeYUdxnasxzBDAtd98MHF1Zp6ST+ecKx9TdhfGWOYfkfeotlr3anw/sXGe+R5DPFcD1pJqPravSfpi7dwL+VjX+qcD/VR9XwA+AKwvDrUWqqSpeMdc9DxXXbwvHxoW5e1Cefu+2ngNo+Yr5CWBEoV8fCufxFdmOzX1WhSvml0mlpubuQW1Cqk6p+GtOa63i07L/JAWL1qo17x6ke+Gt9aGIWL/wuQWWPjxzg6S/S3qddBW4UR6nH6lAUk/1Mq1RZx2+TNoJ69kIWJ3ll7FYMn6x0P0WQERUpxXX6exKR0S8QaqKr7e96m4bSf8h6Yn8YNFrpBL7RtUTqKM12601+0Vr1k9dktYk3Xu9AiAi7iNdSRxWyGdxfQUpIFZsBpwj6bW8Dl4hBc3l5i+pu6QzJT2b96dZhWWoqLfMtfIxm/pa2q8qitPYDDiosix5eXbN09kEeDUi3iwMX1znK2oj0jJ+h3TyXq3OcEeRnr14UtKDkvat6n8ScF5ENPe09H3AlpI2Jl3lXQr0y7URw2nbPd6XI2Jx4Xdz561JwHb5inpnUkHvSaBPTtu1MO/NgE2qtsFp1D6fVe8T/yRt96LWnofqGQMcnK+CvwjcHIXnNd7lOaBoM+DawjI/QbpgqLXc73o7rgqB+T7SvaL9mxlmLmnFVvTPaZDuzaxV6SGp7pOxK6jWvBezbLBaUReQ7ukNilSFdhrppAvpgFiR6vFqE4C+kobV6f8SqeRYvYwvvIt59qt05CruDWnaXq2Sq3//EzgY2CAi1ifd+6ysn2hhEu213d7t+jkAeA9wfi6A/Z0UVCvV2fNI1dxAeoK++Ju0H3y1qmC3ZkTcW2Neh5EemNyddAIbUJlsK/I5j2W3m4q/a7iNdL+1JcXtNJt0xVxclrUj4sw8/w0krV0Yvn8rp9t8BiKWRMTPSeeZb9QZ5pmIOJR0C+C/gT9U5WVP4HuS6i5zDlwPkR60fDwi3ibdTz+RdC/1pdbmuS0i/cthLnAM6Wr4jdzrvpy2DnB/TpsNPF+1DdaNiH1qTLp631yTdLul1VlrRd7vIgX7/Ui3xZZWY7fiHFBUHQe6k279VMwGPlW13GtEegitOk/vejuu9IE5Iv5BqjI5T9L+ktaStJqkT0mq3Je4knRQ9Mqllh+QHj6BdK9kW0lDc6nr9DZm4UXSvcN6rgS+nf/usg5N9zYXNzNOa61LquZ+Q9LWpIeDKm4A3ifpBEk9Ja0raae2ziAiniHdJ7pS6f+Rq0taQ9JISadE+vvJOGB0nsdmpB3w3fw/cB9Ju0panXSv8IGIaO7qq5Z1SYF0AdBD0g9IAa7iRWBAvmdWS7tst3ZYP6OAi0m3AIbmzy7AUKW/q90IfCDv+z2AY0n3Fyt+A5wqaVsASetJOqjOvNYlVbu/TDpJ/aT1S8qNpOPoczkfx1flo9oPgY9I+lmlMCxpoKTLJa1fZ5zLgc9I2itf3a+R98m+EfFX0sNXZ+R9dFfgM83Mv6XjtpYzgZNr3aOU9AVJvSLiHVL1N6QrqorppIcOz5P02WbmcSdwHE33ISdW/a5lRZal2l2k/fKuQtrdOW1KRLyV0yYDr0v6T0lr5u0wRNKONab5B9L2+kg+ls+gdYW8itYu16WkwtD6pPvfFS2dA4qeJl2tf1rSaqRnJYr/fvgN6RjeDCDHkub+9bMi23GplT4wA0TEL0g70PdIG2E2aSVclwf5MemgnQY8Rnqo5cd53KeB/yKV4J8h7YxtcTowJldxHFyj/8WkJwwnkZ5g/BfwzTbO41Et+z/mX+X075CuchYCvwWWPnwWEQtJ94o+Q6oueob0INOKOJ70UMZ5pJPOs6QrucpB8E1SifM50vr7PWm5V9TvSSfuV4AdSM8RtNUtpPurT5OqNP/FstWiV+fvlyU9XGP89thuFSu0fpQedhoB/Coi/l74PES6dzUql74PAs4iBdTBpH19EUBEXEs6aY3N1dOPkx44q+VS0rp6gXQ/9P46wy2nkI8zcz4GkZ7Grzf8s6T75QOA6ZL+QXp4bAppf641zmzSldFpNB3nJ9F0HjuMdA/0FdL+c2mNyVScAxwo6VVJ/9OqhUyFj1eBr9Tot3dejjfytEdGxL+q8v8osC/wW0n1tsGdpIAyqc7vWk6n+XNQa9xJutovnv/uymlL550Lmp8hFRCfJ9UIXUSqYVlGREwn7ftjSVfPC0n3uhe1Mk8/JV1QvSbpO80MdympduSqiChOu6VzQDGv/yDVhlxE2v/fZNlbQueQHk77i6SFpGOjuQudFdmOSynfkDazVUCuAZgDHB4Rd3R2fswqcs3Ta6Rbb893dn7KbJW4YjbrynLV7vpKDY9UnjNo9dWuWaNI+ky+vbg26an2x2h6qNDqcGA2W/l9mHR74SVSNeP+hXuCZp1pP5oadhpEquJ3NW0LXJVtZmZWIr5iNjMzKxEHZjMzsxLprDe2tIuNNtooBgwY0NnZMDMza5OHHnropYjoVavfSh2YBwwYwJQpUzo7G2ZmZm0iqW6Tsa7KNjMzKxEHZjMzsxJxYDYzMysRB2YzM7MScWBeiS1ZsoTtt9+effdNr3496aST2Hrrrdluu+044IADeO219JKbK664gqFDhy79dOvWjalTp3Zm1s3MrA4H5pXYOeecwzbbbLP09x577MHjjz/OtGnT2HLLLfnpT38KwOGHH87UqVOZOnUql112GQMGDGDo0KGdlW0zM2uGA/NKas6cOdx4440cffTRS9P23HNPevRI/4DbeeedmTNnznLjXXnllRx66KEdlk8zM2sbB+aV1AknnMBZZ51Ft261N+HFF1/Mpz61/Ctfr7rqKgdmM7MSc2BeCd1www307t2bHXbYoWb/0aNH06NHDw4//PBl0h944AHWWmsthgwZ0hHZNDOzFbBSt/zVVd1zzz2MHz+em266iX/961+8/vrrfOELX+Dyyy9nzJgx3HDDDUyYMAFJy4w3duxYXy2bmZXcSv3ax2HDhkVXb5Jz4sSJnH322dxwww3cfPPNnHjiidx555306rVsE6zvvPMO/fv3Z9KkSWyxxRadlFszMwOQ9FBEDKvVz1XZq5DjjjuOhQsXssceezB06FC+9rWvLe03adIk+vbt66BsZlZyvmIuGHDKje02ra5u1pmf7uwsmJmVlq+YzczMVhIOzGZmZiXiwGxmZlYiDsxmZmYl4sBsZmZWIg7MZmZmJeLAbGZmViIOzGZmZiXiwGxmZlYiDsxmZmYl4sBsZmZWIg7MZmZmJdLwwCypu6RHJN2Qf28o6VZJz+TvDQrDnipppqSnJO3V6LyZmZmVTUdcMX8LeKLw+xRgQkQMAibk30gaDIwEtgX2Bs6X1L0D8mdmZlYaDQ3MkvoCnwYuKiTvB4zJ3WOA/QvpYyNiUUQ8D8wEhjcyf2ZmZmXT6CvmXwEnA+8U0jaOiHkA+bt3Tt8UmF0Ybk5OMzMz6zIaFpgl7QvMj4iHWjtKjbSoMd1jJE2RNGXBggXvKo9mZmZl08gr5l2Az0qaBYwFPinpcuBFSX0A8vf8PPwcoF9h/L7A3OqJRsSFETEsIob16tWrgdk3MzPreA0LzBFxakT0jYgBpIe6bo+ILwDjgVF5sFHA9bl7PDBSUk9JmwODgMmNyp+ZmVkZ9eiEeZ4JjJN0FPA34CCAiJguaRwwA1gMHBsRSzohf2ZmZp2mQwJzREwEJubul4ERdYYbDYzuiDyZmZmVkVv+MjMzKxEHZjMzsxJxYDYzMysRB2YzM7MScWA2MzMrEQdmMzOzEnFgNjMzKxEHZjMzsxJxYDYzMysRB2YzM7MScWA2MzMrEQdmMzOzEnFgNjMzKxEHZjMzsxJxYDYzMysRB2YzM7MScWA2MzMrEQdmMzOzEnFgNjMzKxEHZjMzsxJxYDYzMysRB2YzM7MScWA2MzMrkYYFZklrSJos6VFJ0yWdkdNPl/SCpKn5s09hnFMlzZT0lKS9GpU3MzOzsurRwGkvAj4ZEW9IWg24W9Kfc79fRsTZxYElDQZGAtsCmwC3SdoyIpY0MI9mZmal0rAr5kjeyD9Xy59oZpT9gLERsSgingdmAsMblT8zM7Myaug9ZkndJU0F5gO3RsQDuddxkqZJuljSBjltU2B2YfQ5Oc3MzKzLaGhgjoglETEU6AsMlzQEuAB4PzAUmAf8PA+uWpOoTpB0jKQpkqYsWLCgQTk3MzPrHB3yVHZEvAZMBPaOiBdzwH4H+C1N1dVzgH6F0foCc2tM68KIGBYRw3r16tXgnJuZmXWsRj6V3UvS+rl7TWB34ElJfQqDHQA8nrvHAyMl9ZS0OTAImNyo/JmZmZVRI5/K7gOMkdSdVAAYFxE3SLpM0lBSNfUs4KsAETFd0jhgBrAYONZPZJuZWVfTsMAcEdOA7Wukf7GZcUYDoxuVJzMzs7Jzy19mZmYl4sBsZmZWIg7MZmZmJeLAbGZmViIOzGZmZiXiwGxmZlYiDsxmZmYl4sBsZmZWIg7MZmZmJeLAbGZmViIOzGZmZiXiwGxmZlYiDsxmZmYl4sBsZmZWIg7MZmZmJeLAbGZmViIOzGZmZiXiwGxmZlYiDsxmZmYl4sBsZmZWIg7MZmZmJeLAbGZmViIOzGZmZiXSsMAsaQ1JkyU9Kmm6pDNy+oaSbpX0TP7eoDDOqZJmSnpK0l6NypuZmVlZNfKKeRHwyYj4IDAU2FvSzsApwISIGARMyL+RNBgYCWwL7A2cL6l7A/NnZmZWOg0LzJG8kX+ulj8B7AeMyeljgP1z937A2IhYFBHPAzOB4Y3Kn5mZWRk19B6zpO6SpgLzgVsj4gFg44iYB5C/e+fBNwVmF0afk9PMzMy6jIYG5ohYEhFDgb7AcElDmhlctSax3EDSMZKmSJqyYMGC9sqqmZlZKXTIU9kR8RowkXTv+EVJfQDy9/w82BygX2G0vsDcGtO6MCKGRcSwXr16NTTfZmZmHa2RT2X3krR+7l4T2B14EhgPjMqDjQKuz93jgZGSekraHBgETG5U/szMzMqoRwOn3QcYk5+s7gaMi4gbJN0HjJN0FPA34CCAiJguaRwwA1gMHBsRSxqYPzMzs9JpWGCOiGnA9jXSXwZG1BlnNDC6UXkyMzMrO7f8ZWZmViIOzGZmZiXiwGxmZlYiDsxmZmYl4sBsZmZWIg7MZmZmJeLAbGZmViIOzGZmZiXiwGxmZlYiDsxmZmYl4sBsZmZWIg7MZmZmJeLAbGZmViIOzGZmZiXiwGxmZlYiDsxmZmYl4sBsZmZWIg7MZmZmJeLAbGZmViIOzGZmZiXiwGxmZlYiDsxmZmYl4sBsZmZWIg7MZmZmJdKwwCypn6Q7JD0habqkb+X00yW9IGlq/uxTGOdUSTMlPSVpr0blzczMrKx6NHDai4H/iIiHJa0LPCTp1tzvlxFxdnFgSYOBkcC2wCbAbZK2jIglDcyjmZlZqTTsijki5kXEw7l7IfAEsGkzo+wHjI2IRRHxPDATGN6o/JmZmZVRh9xjljQA2B54ICcdJ2mapIslbZDTNgVmF0abQ41ALukYSVMkTVmwYEEDc21mZtbxGh6YJa0DXAOcEBGvAxcA7weGAvOAn1cGrTF6LJcQcWFEDIuIYb169WpQrs3MzDpHQwOzpNVIQfmKiPgjQES8GBFLIuId4Lc0VVfPAfoVRu8LzG1k/szMzMqmkU9lC/gd8ERE/KKQ3qcw2AHA47l7PDBSUk9JmwODgMmNyp+ZmVkZteqpbEm7RMQ9LaVV2QX4IvCYpKk57TTgUElDSdXUs4CvAkTEdEnjgBmkJ7qP9RPZZmbW1bT271K/Bj7UirSlIuJuat83vqmZcUYDo1uZJzMzs1VOs4FZ0oeBjwC9JJ1Y6PUeoHsjM2ZmZtYVtXTFvDqwTh5u3UL668CBjcqUmZlZV9VsYI6IO4E7JV0SEX/toDyZmZl1Wa29x9xT0oXAgOI4EfHJRmTKzMysq2ptYL4a+A1wEeAnpc3MzBqktYF5cURc0NCcmJmZWasbGPmTpG9I6iNpw8qnoTkzMzPrglp7xTwqf59USAtgi/bNjpmZWdfWqsAcEZs3OiNmZmbW+iY5v1QrPSIubd/smJmZdW2trcresdC9BjACeBhwYDYzM2tHra3K/mbxt6T1gMsakiMzM7MubEVf+/hP0msZzczMrB219h7zn0hPYUN6ecU2wLhGZcrMzKyrau095rML3YuBv0bEnAbkx8zMrEtrVVV2fpnFk6Q3TG0AvN3ITJmZmXVVrQrMkg4GJgMHAQcDD0jyax/NzMzaWWursr8L7BgR8wEk9QJuA/7QqIyZmZl1Ra19KrtbJShnL7dhXDMzM2ul1l4x3yzpFuDK/PsQ4KbGZMnMzKzrajYwSxoIbBwRJ0n6HLArIOA+4IoOyJ+ZmVmX0lJ19K+AhQAR8ceIODEivk26Wv5VozNnZmbW1bQUmAdExLTqxIiYAgxoSI7MzMy6sJYC8xrN9FuzuREl9ZN0h6QnJE2X9K2cvqGkWyU9k783KIxzqqSZkp6StFfrF8PMzGzV0FJgflDSV6oTJR0FPNTCuIuB/4iIbYCdgWMlDQZOASZExCBgQv5N7jcS2BbYGzhfUve2LIyZmdnKrqWnsk8ArpV0OE2BeBiwOnBAcyNGxDxgXu5eKOkJYFNgP2C3PNgYYCLwnzl9bEQsAp6XNBMYTnrQzMzMrEtoNjBHxIvARyR9AhiSk2+MiNvbMhNJA4DtgQdIT3lXAvY8Sb3zYJsC9xdGm5PTqqd1DHAMQP/+/duSDTMzs9Jr7fuY7wDuWJEZSFoHuAY4ISJel1R30FqzrpGXC4ELAYYNG7ZcfzMzs5VZQ1vvkrQaKShfERF/zMkvSuqT+/cBKi2KzQH6FUbvC8xtZP7MzMzKpmGBWenS+HfAExHxi0Kv8cCo3D0KuL6QPlJST0mbA4NIL84wMzPrMlrbJOeK2AX4IvCYpKk57TTgTGBcfrL7b6Q3VhER0yWNA2aQnug+NiKWNDB/ZmZmpdOwwBwRd1P7vjHAiDrjjAZGNypPZmZmZec3RJmZmZWIA7OZmVmJODCbmZmViAOzmZlZiTgwm5mZlYgDs5mZWYk4MJuZmZWIA7OZmVmJODCbmZmViAOzmZlZiTgwm5mZlYgDs5mZWYk4MJtZl3HkkUfSu3dvhgwZskz6r3/9a7baaiu23XZbTj755KXpP/3pTxk4cCBbbbUVt9xyS0dn17qoRr720cysVI444giOO+44vvSlLy1Nu+OOO7j++uuZNm0aPXv2ZP78+QDMmDGDsWPHMn36dObOncvuu+/O008/Tffu3Tsr+9ZF+IrZzLqMj33sY2y44YbLpF1wwQWccsop9OzZE4DevXsDcP311zNy5Eh69uzJ5ptvzsCBA5k8eXKH59m6HgdmM+vSnn76ae666y522mknPv7xj/Pggw8C8MILL9CvX7+lw/Xt25cXXnihs7JpXYirss2sS1u8eDGvvvoq999/Pw8++CAHH3wwzz33HBGx3LCSOiGH1tX4itnMurS+ffvyuc99DkkMHz6cbt268dJLL9G3b19mz569dLg5c+awySabdGJOratwYDazLm3//ffn9ttvB1K19ttvv81GG23EZz/7WcaOHcuiRYt4/vnneeaZZxg+fHgn59a6Aldlm1mXceihhzJx4sSlV8RnnHEGRx55JEceeSRDhgxh9dVXZ8yYMUhi22235eCDD2bw4MH06NGD8847z09kW4dQrfsoK4thw4bFlClT2m16A065sd2m1dXNOvPTnZ0F60A+dtqHj5uuQ9JDETGsVj9XZZuZmZWIA7OZmVmJNCwwS7pY0nxJjxfSTpf0gqSp+bNPod+pkmZKekrSXo3Kl5mZWZk18or5EmDvGum/jIih+XMTgKTBwEhg2zzO+ZL8lIWZmXU5DQvMETEJeKWVg+8HjI2IRRHxPDAT8P8SzMysy+mMe8zHSZqWq7o3yGmbArMLw8zJacuRdIykKZKmLFiwoNF5NTMz61AdHZgvAN4PDAXmAT/P6bXauav5P66IuDAihkXEsF69ejUml2ZmZp2kQwNzRLwYEUsi4h3gtzRVV88B+hUG7QvM7ci8mZmZlUGHBmZJfQo/DwAqT2yPB0ZK6ilpc2AQ4PermZlZl9OwJjklXQnsBmwkaQ7wQ2A3SUNJ1dSzgK8CRMR0SeOAGcBi4NiIWNKovJmZmZVVwwJzRBxaI/l3zQw/GhjdqPyYmZmtDNzyl5mZWYk4MJuZmZWIA7OZmVmJODCbmZmViAOzmZlZiTgwm5mZlYgDs5mZWYk4MJuZmZWIA7OZmVmJODCbmZmViAOzmZlZiTgwm5mZlYgDs5mZWYk4MJuZmZWIA7OZmVmJODCbmZmViAOzmZlZiTgwm5mZlYgDs5mZWYk4MIulspIAAB1DSURBVJuZmZWIA7OZmVmJODCbmZmViAOzmZlZiTQsMEu6WNJ8SY8X0jaUdKukZ/L3BoV+p0qaKekpSXs1Kl9mZmZl1sgr5kuAvavSTgEmRMQgYEL+jaTBwEhg2zzO+ZK6NzBvZmZmpdSwwBwRk4BXqpL3A8bk7jHA/oX0sRGxKCKeB2YCwxuVNzMzs7Lq6HvMG0fEPID83TunbwrMLgw3J6eZmZl1KWV5+Es10qLmgNIxkqZImrJgwYIGZ8vMzKxjdXRgflFSH4D8PT+nzwH6FYbrC8ytNYGIuDAihkXEsF69ejU0s2ZmZh2towPzeGBU7h4FXF9IHympp6TNgUHA5A7Om5mZWafr0agJS7oS2A3YSNIc4IfAmcA4SUcBfwMOAoiI6ZLGATOAxcCxEbGkUXkzMzMrq4YF5og4tE6vEXWGHw2MblR+zMzMVgZlefjLzMzMcGA2MzMrFQdmMzOzEnFgNjMzKxEHZjMzsxJxYDYzMysRB2YzM7MScWA2MzMrEQdmMzOzEnFgNjMzKxEHZjMzsxJxYDZrZ6+99hoHHnggW2+9Ndtssw333Xcf3//+99luu+0YOnQoe+65J3Pn1nyrqZmZA7NZe/vWt77F3nvvzZNPPsmjjz7KNttsw0knncS0adOYOnUq++67L//1X//V2dk0s5Jq2NulzLqi119/nUmTJnHJJZcAsPrqq7P66qsvM8ybb76JpE7InZmtDByYzdrRc889R69evfjyl7/Mo48+yg477MA555zD2muvzXe/+10uvfRS1ltvPe64447OzqqZlZSrss3a0eLFi3n44Yf5+te/ziOPPMLaa6/NmWeeCcDo0aOZPXs2hx9+OOeee24n59TMysqB2awd9e3bl759+7LTTjsBcOCBB/Lwww8vM8xhhx3GNddc0xnZM7OVgAOzWTt63/veR79+/XjqqacAmDBhAoMHD+aZZ55ZOsz48ePZeuutOyuLZlZyvsdsK4UBp9zY2VlotbcHHsx2u+1LLFlMj/Xfx3v3OYHz9z2Cf78yB9SNHu/pxYZ7HdtpyzTrzE93ynzNrHUcmM3a2eobb0GfUb9aJq3XAad1Um7MVh4DBgxg3XXXpXv37vTo0YMpU6bwyiuvcMghhzBr1iwGDBjAuHHj2GCDDTo7qw3lqmwzMyuNO+64g6lTpzJlyhQAzjzzTEaMGMEzzzzDiBEjlj5MuSpzYDYzs9K6/vrrGTVqFACjRo3iuuuu6+QcNZ4Ds5mZlYIk9txzT3bYYQcuvPBCAF588UX69OkDQJ8+fZg/f35nZrFDdMo9ZkmzgIXAEmBxRAyTtCFwFTAAmAUcHBGvdkb+zMys491zzz1ssskmzJ8/nz322KPL/nuhM6+YPxERQyNiWP59CjAhIgYBE/JvMzPrIjbZZBMAevfuzQEHHMDkyZPZeOONmTdvHgDz5s2jd+/enZnFDlGmquz9gDG5ewywfyfmxczMOtCbb77JwoULl3b/5S9/YciQIXz2s59lzJgUGsaMGcN+++3XmdnsEJ31d6kA/iIpgP+NiAuBjSNiHkBEzJO06heLzMwMSPeSDzjgACA1bXvYYYex9957s+OOO3LwwQfzu9/9jv79+3P11Vd3ck4br7MC8y4RMTcH31slPdnaESUdAxwD0L9//0blz8xslbAyNc7Dp36ytPO3C+G3lbzveCLsCM8CHzrrvs7JGx3XOE+nVGVHxNz8PR+4FhgOvCipD0D+rvnoXURcGBHDImJYr169OirLZmZmHaLDA7OktSWtW+kG9gQeB8YDo/Jgo4DrOzpvZmZmna0zqrI3Bq7NL4rvAfw+Im6W9CAwTtJRwN+Agzohb2ZmZp2qwwNzRDwHfLBG+svAiI7Oj5mZWZmU6e9SZmZmXZ4Ds5mZWYk4MJuZmZWIA7OZmVmJODCbmZmViAOzmZlZiTgwm5mZlYgDs5mZWYk4MJuZmZWIA7OZmVmJODCbmZmViAOzmZlZiTgwm5mZlYgDs5mZWYk4MJuZmZWIA7OZmVmJODCbmZmViAOzmZlZiTgwm5mZlYgDs5mZWYk4MJuZmZWIA7OZmVmJODCbmZmVSOkCs6S9JT0laaakUzo7P2ZmZh2pVIFZUnfgPOBTwGDgUEmDOzdXZmZmHadUgRkYDsyMiOci4m1gLLBfJ+fJzMysw5QtMG8KzC78npPTzMzMuoQenZ2BKqqRFssMIB0DHJN/viHpqYbnqlw2Al7q7Ey0RP/d2TnoNKXfPt425eVtU27tvH02q9ejbIF5DtCv8LsvMLc4QERcCFzYkZkqE0lTImJYZ+fDavP2KS9vm/LytllW2aqyHwQGSdpc0urASGB8J+fJzMysw5TqijkiFks6DrgF6A5cHBHTOzlbZmZmHaZUgRkgIm4CbursfJRYl63GX0l4+5SXt015edsUKCJaHsrMzMw6RNnuMZuZmXVpDsx1SHqjTvolkg7swHxMlDQsd0vS7ZLe007T7inpqtz86QOSBtQZ7hBJ0yRNl3RWVb+DJc3I/X6f03pJurk98rgqKe5TkvpIuqEdp31c3o4haaNmhrtZ0mvV85Y0VtKg9sqPdU31zpsrMn4DjpHN83numXzeW73OcP0l/UXSE/ncNiCnj5D0sKSpku6WNDCn7yvpjPbKJzgwr2z2AR6NiNfbaXpHAa9GxEDgl8By/9KT9F7gZ8CIiNgW2FjSiNxvEHAqsEvudwJARCwA5knapZ3yuSo6EfhtO07vHmB34K8tDPcz4Is10i8ATm7H/ABtL+AWC6IrOL9GFmQ/lk/Mi+sVziWtJelGSU/mwuqZhX5HSFqQT+xTJR2d012Qra29j5H/Bn4ZEYOAV0nnv1ouBX4WEduQWqOcn9MvAA6PiKHA74Hv5fQbgc9KWqu9MtrlA7OkEyU9nj8n1OgvSefmktONQO8609kxX1XeJ+lnkh7P6WtI+j9Jj0l6RNInWkhfM1+9TJN0FbBmYTaHA9fn4U6WdHzu/qWk23P3CEmXt3Lx9wPG5O4/ACMkVTfysgXwdA62ALcBn8/dXwHOi4hXASJifmG863J+W0XSjyR9q/B7tKTj8/r/Wd4+j0k6JPffrViaztvoiNw9S9IZ+ST6mKStc3ovSbfm9P+V9NdaV5eSjpL0dD7J/1bSuTl9M0kT8raZIKl/C+mb5/3hQUk/qprN54Gb83A3Sdoudz8i6QeFdXJ0a9ZfRDwSEbNaMdwEYGGNXncBu0sq3QOh70J7F2T/BhxBOik35+yI2BrYHthF0qcK/a6KiKH5cxG8+4JsmY6dPOzP83ATJPXKaV/Jx8Gjkq5RDmIddYzk89onSec5SOe9/WsMNxjoERG3AkTEGxHxz9w7gEohbz1yGxuRHtSaCOzbUj5aq0sHZkk7AF8GdgJ2Br4iafuqwQ4AtgI+QApEH6kzuf8DvhYRHwaWFNKPBYiIDwCHAmMkrdFM+teBf0bEdsBoYIfCtHYBHsrdk4CP5u5hwDqSVgN2JZ1kUaqumVrj86U83tImUCNiMfAP4L1VyzUT2FrSgHzS3p+mRmC2BLaUdI+k+yXtXRhvSiF/rfE7YFTOdzfSf9ivAD4HDAU+SLoi/JmkPq2Y3ksR8SFSKfc7Oe2HwO05/Vqgf/VIkjYBvk/aH/YAti70Phe4NG+bK4D/aSH9HOCCiNgR+HthHpuTaioW5aRJwEeVruwWk7Yz5G0pad0623Gq2uklLxHxDmlbf3BFxlc7FXCzL0i6N09reB5/eE57JH9vldM7rCAbEbMiYhrwTjPD/DMi7sjdbwMPkxpKakmbCrJVSnHsZGsDD+fh7szjAfwxInaMiA8CT9B0tdpRx8h7gdfyeQ7qN/e8JfCapD/mfe1nSi9XAjgauEnSHFKt05mF8dp6vmtWlw7MpI16bUS8GRFvAH9k+ZX7MeDKiFgSEXOB26snIml9YN2IuDcnFUvUuwKXAUTEk6Sqxi2bSf8YcHlOnwZMK0xrw4ioXO08BOwgaV1gEXAfKUB/lByYI+KQQum8+Lm0kvUa62SZx/Tz1fDXgavydGeRDgxIf7cbBOxGKlxclNcFpOqfTWpMv6Z8tfdyLhjtCTwSES+T1lNl/b9IOth3bMUk/5i/HwIG5O5dSS9GISJuJlVnVRsO3BkRr0TEv4GrC/0+TNO2vSxPr7n0XYArC+kVfYAFhd93kbb7rqRqsXXyFcWAiHgqIhbW2Y5DI2JGSyuiDdq0zSrauYALsHZEfAT4BnBxTnsS+FhEbA/8APhJTu/Igmyb5GPhM8CEQvLncyHiD5KKrRyu8Im9RMcOpELLVbn7cpqOhSGS7pL0GKkAsm1O76hjpMVzXdaDtB2+Q1pXW5BqSQC+DewTEX1JF2K/KIy3QsdOPatStdWKqLWxamnpP2XNTadev+bGqTe/xZK6RcQ7EfFvSbNIJ8R7SQH8E8D7SSVS8hXEVjWm84scnCtNoM7JV8PrAa8sl5mIPwF/ytM8hqYagTnA/TmAPa/UbvkgUgtuawBvNbOMtVxEOgjeR9MJud56WsyyBcs1qvpXStpLaNrPW7O9W7tPQP3tFC0M8xbL5vdBUrB4DriV1G7wV8hBJRe+7qozr8PaMTivyDaDQgEXQFKlgPtIYZilBVxgbuWKtY4rASJikqT3VAq+pFqlQaR1ulphuv+Th58mqbUF2YdpKsgen8c/ZAWWvaZ8PF0J/E9EPJeT/0RaB4skfY1UnfrJ3O/dntjLcOzUUtn/LwH2j4hHlarNd6sxTFG7HiOkc+L6knrkq+blmnvO5pAKNs/l6V4H7CxpPPDBiHggD3cVuZo9W9Fjp6aufsU8Cdhf6YGNtUml+uqNOwkYKal7rgb6RPVE8lXlQkk756SRVeMfDiBpS1IV0FOtTB8CbFeY1lOkElxx2t/J33cBXwOm5nserbliHk+uAgMOJFVVLXeQSOqdvzcgXcVclHtdV1kfSvebtiQdOOTux6un1YJrgb1JJdVbCst4SF7/vUgn4smkGobBSk+WrweMaMX07wYOzvndE9igxjCTgY9L2iCfXD9f6HcvTdv28Dy95tLvqUqveJqmK5FKlefsnLf7SdvyOzTVfHTUFfOWwIq0tNdeBdx6wwXwI+COiBhCugpdo5nhKxbnql1y4XEWTQXZu6hRkG3HK+YLgWci4ldLMxnxcqFq9rcse3X/bk/sZTh2IMWUyoNxh9F0LKxLuo++GsseCx1yjOTz2h2FvI0i3+ao8iCwQV5fkApOM0g1BOvlczWk21xPFMZbkfNdfRHRpT+kJ/8ez58TCulv5G+R7iHOIAWi64ADa0xnJ9JV633AT4F7cvoapNLiY6QriE+0kL4mqcpoGunpwHuBYbnf94GjC/McAfybVPUHaWc+sQ3LvgapqnYm6YDdotBvaqH7yrz8M4CRhXSRqnNm5OUo9vsO8M0V2B6/Ac6smsfP8vZ5DDik0O8sUmHlBlL12xE5fRawUe4eBkzM3b1J1YoPk55Cnwv0rJGHY/K6nEi6zzY6pw8g3cqYlqfTv4X0zfP+8CBwSmWfyv0mAAMLv38E3Ju7NyEFmw+1Yb0dTyrtL87LdVFh+S8qDHcXqYrwrTz8Xjl9Y2DyCh5DH8rLvhbpHuPjwPZVx9HnaGpqtw/pRFfrOJoI/CZ37wo8lruvBT6fu08HZhWO38qyDsnLXzle7q9ax6eTHuDaPS/v30hX+m1d3ktq5b3Q/8fANUC3qvQ+he4DSLVNld87ADevyPov2bHzRt6XHyIdE71y+teB5/P2/TVwSSccI1uQznMzSee9nnWOkT1I+/NjeVuvXthmjwGP5uUoni9vAD7wbrbfMnltrwl19Q+wTqH7FOCcBsyjD3BrZy9rK/M6CdigjeN0A6YCgxqUp56kJy4h3ReeWme4dfJ3D1L14wENyMsBwI87ezsV8vNt4Kh3MX57FXAnkgq29+ZpDS9sr6dJV1g/oikwd2RBdkdSYeZN4GVgeqHf1PzdNweMJ/K+PLWSh7xc0/OJ/Q5g68L4K1SQLYxfimOnnedZqmOkmXxuDExoz2m6Sc52ovRXhFNJJ/O/kkqgC5ofa4XmczCpZN1efwFpd7kaaJeIuK4N4wwmlTqvjYj/aFC+BgHjSCext4FvRMSDNYY7m3RVtQbwF+Bb0YADRdLRkf8y09kkfRm4LJqeWl3p5VtPl0bEHp2dl5ZImgTsF/mvh20ctzTHTgPmW5pjpB5JOwL/joip7TZNB2YzW1WtqgVZW7U5MJuZmZVIV38q28zMrFQcmM3MzErEgbkEJL0vNyv4rFKThTcV/i/3bqZb70UBwyT9T61xWjnd06p+31tv2EbJ/8G8Lf/P9JBC+nk5bYaktwr/RW23N4JJ+rSkh/I8npS03Ms/qob/ZOE/7g0l6Zbc2EKnkfRj1WiWM/c7Qqmpzen58+2Ozl8jSJojaf38n+F6DV20dZqP5bYMkLRa3p9HFvo/qtx+dJ3xd5L0yxbmMVBSzYeWJB0p6X010o+WdFlV2saS5uf/Kdeb1+WSlmuf2pbnwNzJJIn0H82JEfH+iBgMnEZ6BL8hImJKRBz/LiaxTGCO1HxiR9seWC1SAwKVJgCJiGMjvf1lH+DZaGpk4A91p9QGkj4I/Ao4NG+rIaT/fjbnk6SmKhtGSbeI2CuaWrsqFUn7AscBu0d6G9kwar9Qo974PZr7XQaRmr9srzaT76Wp6dIPkf6C9RFY2tJVP5pp1CIiHoiId1PwOZLUkli1a4BPKbXtX3EQ6anwf7+L+VlFZ/8HrKt/SCftSXX61WwggNSc3Z2kvy88TWpM/XDSn+cfA96fh7uE1OjAXXm4fQvj35C7Tyc14TeR1GrX8YX5X0dqKGA6cExOO5PUVN9U4Iqc9kYr8juR9GaXJ0kN7KswvRmk/6GeXWMdbJjzMY3UYMR2pMYOZpJeujG1srxV4w0AHi/83gR4IHfvQPqv6Sb59/Okv0ZtTvp/6TRSs399a0z398CX6myv/YAHSA3G/CXn8/2kxvlfyHn9CKnQ9UdS+8iTgZ3z+MVGHM7P46yf+51M0/+Ev5nTBubfv8nz3JT0P9vKOKPy9Kfm6XUj/Z3vsrx9Hi9u7+aWI6f/mPTChDvzvnJsYZwfkBqsuJXUXOEJNaZ7L6m961rr7kN5ntNIJ/71cvrdpDawJ5FeK3o58PO8nc4C1iHt55Nzfj+Tx/sAqdGKqXmaW+T1Nb2w/OOANfPwe+RhHyO1ylVpVGIO6Rh5JE9ny5zeKy/rw6RGaF4A1s/r97U8zO55e/4xr5tLC8v72Zx2F6nBjetqrJMv0dQQx7dJL1G4u5DfW3J3vXWwe2W61Nm3aNqHfpfXzZ9Jx8IhpMZCnsrrZfWqvI0nN/hS2E675e4z8rqv7JuVY/1yUrOclfVa2U93Bm5rYVmW256dfe5u5KfTM9DVP6QWm35Zp9/n88HfnaaWivqQAt1rubtnPsjOyON8C/hV7r6E1J5rN1Ib1nPyQbcbywbme/N0NiI1nLBa7rdh/l4zH2Tvzb/fqMrnG63I7z9IjS90I7X0sysp6D5VOHDXr7EOfg38MHd/kqaGHJYuQ511N4BCYM5pT5JapjohH+SHkALnXbn/n0nvW4XU+tcfakx3GrBtnXluUFiWrwH/nbt/zLKNblxFUzBemk/SSeyk3L0vqfCwPunFGo+SWtZal3TltB3ppPoOsGNh2nPyOENIBZpKoxAXkppI3An4c2H4Wuu8ueW4C1iddKJ/OW/rSv7WJLW3/jy1A/M/yI171Og3A9g1d/+EXEgjnfB/XRju8rxc3fLvs8gtzuV8P03axy+gqWDYM6cNzOu0su4vzfvCWqTmHisF2iuA4wrr8+uFY7XSKtn5wGm5e7/CtqoOzK+SjoHupH1u5zy/OcBmpMLs1dQOzANJr1wlDzOQVEBZi/TWph+0sA6KgbnevjWQ1OjKB3K/PxamdTcwtM72OhS4Onf3y8tT2SaV84ZIrQZ+qrDtWgrMrd6e7XkeLtvHVdnl1tzbYR6MiHmR2t59lnRlA6nEP6AwjXGRXnrxDOkqp/gaw4obI2JRRLxEaky/Uo1+vKRHSVeq/UjBfUXzOzki5kR6veDUnMfXgX+R3kr1OeCfdaZZeQvX7cB7c/u+K+I+0hXrR0kn/49ReBsXKWiNzd2X0va3/fQH/qL0Bp0TaXqDTrXdgd/ke3vXkdrmXZNl3+BzA03VvB8Fron0SsGFeZzKW3uejdoNPexOWvdT8nw+TiqEzAS2knSOpL1IwbIty3FDRLwd6d3br5CuHD+W8/dWRPyD/MKT1pL0XtKJttKu8pg8zYqxVaNcnfcjSG9T+m5exjtIJ/H+pMLm9ySdDPSLiH/l4Z+PiPtzd+XtR9uQ2rV+NqdfWjX/Wm9bKr4F7nrqV8nfn4/TSi3TAGAw8FRE/DVSpLmy1ogRMRN4j1I79APz7ymkgtBH8jI2tw6K6u1bADMj4rEay9ic8cBuktYhFXDHFbbJCEmTSYW1j1P/OKilrdtzleTA3Pmms2xj9kXNvRxgUaH7ncLvd1j2rWFRNV717+ppLQF6SNqNdHL/cKR3qD7C8m+hWdH8LiFdyS0mnWSuIb3n+eYa47X2dW2tUXl13Kak4LE96YQ1qQ3TaG57nUeq/fgA6WUf9daXSE1NVu5/bxoRb7FibyJ7s5lxLi7MY6uI+FGk1wFuR7oaOh743zYux3LbMXe3ZpvMoPa6a+klGNXLWPwt0lVYZTn7R8TTEXEZqUnHRcCtkiqBttbx0NL8a71tqda0mhu3OH5b3tR0H+nqdHb+fT9pnx1GqvqHOuugajptOjZbylSkN4ndRqotGEkuXCi9ivFcUjO225Fuk9U6DopvuCr2b+v2XCU5MHe+24Gekr5SSZC0o6SPU//tMG1xkKRukt5Pus/2VCvHW4/0ovJ/StqaZR9e+nedpy/blN9c2l4vIm4iVSkOrTPNytu2diO9xH1FW3GaRLrv+mQuFCwkldArVx73k9+gA3yB2gH7LFLJfWDOU3dJJ+Z+6wEv5Af6RhXGWUiqgq64DTi28kNSZbmLb/DZpzDOJOAASWvmdbYf9V9xV5zHwflqC0nvldQ/bxdFxNWk6tAP1Ri33nLUMwn4nKQ1lF5kv2+d4X4KnC1p45ynNSR9M9fUvCWp8qDTF0m1La1xC/m1jXma2+fvLSJiZkScQ3p/b+Xp5c2VmlCEFPDuJhUYBkmqvLntC62Yf3G//AzLbt+WTCfVWvTL67i5103eQzo27su/7yO9Ietv0fSQX811UKXevtWc6v222pXASaQq6UqtzZqki4OX8gNqn68z7iyaCmnFYdq6PVdJDsydLFdlHQDsofR3qemk+75zSU9rTyNVCd0OnBwRf2/jLJ4inWT+DHytDVVAN5OunKeRXhpwf6HfhcA0SVdUjdPW/K4L3JDncSfpAZdqpwPD8jBn0rpAUVOuCuxBU8C9h2UD/XHAMXleh9TKT0Q8QnrhwDhJT5BuHVReEXc6aR3cCbxYGO16UpB8JAefY4FdJE2TNIP0XllIgfLTkh4m3U9/EXgzIiaTToIPkrbDBYWqx3rL+hjpIZzb8vL8hXSLoh8wKVcV/paqJ+xbWI5685qch3+UdC+0Zg1ERIwnXaHfnvfzKTSdg74I/DLndTDpfnZrnAGslf9aVDl2AA5T+jvWVFKB9PKcPh34Sp7P2sCFEfFP4Cjgj7n6fhFp3TTnh8DueVvtRnrOo1Xy/I4jFZ7uIh3rtW4pQNpHtyAH5oiYTbrHWvyLYr11UJ3f5fatFrL6f6TbTFMlrV6j/82kaualtxpyjcwY0jMp19J0VV/tdOB8pb+Wvd2KZam3PVdJbpLTrCTy308WR8RiSbuSHuIb1tn5WlXkWo4/RPo7XWfnZZ2IeCNfMf8v6fWWv27g/LxvrURK9z9Asy5sAHClpO6kq7avdm52rIG+Lulw0tXvFFq+Qn+3BuB9a6XhK2YzM7MS8T1mMzOzEnFgNjMzKxEHZjMzsxJxYDYzMysRB2YzM7MScWA2MzMrkf8HcROJUXGy5WQAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "old_good = df[(df['age']==1.0)&(df['credit']==1.0)].shape[0]\n", + "young_good = df[(df['age']==0.0)&(df['credit']==1.0)].shape[0]\n", + "old_bad = df[(df['age']==1.0)&(df['credit']==2.0)].shape[0]\n", + "young_bad = df[(df['age']==0.0)&(df['credit']==2.0)].shape[0]\n", + "\n", + "import matplotlib.pyplot as plt\n", + "fig = plt.figure()\n", + "ax = fig.add_axes([0,0,1,1])\n", + "langs = ['old good(w=0.96)', 'young good(w=1.1)', 'old bad(w=1.25)', 'young bad(w=0.68)']\n", + "counts = [old_good,young_good,old_bad,young_bad]\n", + "ax.bar(langs,counts)\n", + "for i, v in enumerate(counts):\n", + " plt.text(langs[i],v, str(v))\n", + "plt.title(\"Count of Each Combination of Age and Credit Risk with Weight Value w\")\n", + "plt.xlabel('Combinations of Two Categories and Corresponding Weight Values')\n", + "plt.ylabel('Count')\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 4. Results with Reweighing" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "After understanding how Reweighing works, we want to see if this preprocessing technique really works. We will use a metric called \"BinaryLabelDatasetMetric.\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We first apply this metric to the original training dataset." + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": {}, + "outputs": [ + { + "data": { + "text/markdown": [ + "#### Original training dataset" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Difference in mean outcomes between unprivileged and privileged groups = -0.169905\n" + ] + } + ], + "source": [ + "metric_orig_train = BinaryLabelDatasetMetric(dataset_orig_train, \n", + " unprivileged_groups=unprivileged_groups,\n", + " privileged_groups=privileged_groups)\n", + "display(Markdown(\"#### Original training dataset\"))\n", + "print(\"Difference in mean outcomes between unprivileged and privileged groups = %f\" % metric_orig_train.mean_difference())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As we can see, the result showed that the privileged group was getting 17% more positive outcomes in the training dataset. Therefore the dataset is biased. We may want to mitigate this bias using the Reweighing preprocessing technique and see what happen to the metric after all." + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": {}, + "outputs": [ + { + "data": { + "text/markdown": [ + "#### Transformed training dataset" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Difference in mean outcomes between unprivileged and privileged groups = 0.000000\n" + ] + } + ], + "source": [ + "metric_transf_train = BinaryLabelDatasetMetric(dataset_transf_train, \n", + " unprivileged_groups=unprivileged_groups,\n", + " privileged_groups=privileged_groups)\n", + "display(Markdown(\"#### Transformed training dataset\"))\n", + "print(\"Difference in mean outcomes between unprivileged and privileged groups = %f\" % metric_transf_train.mean_difference())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As a result, after we apply the Reweighing, the bias is mitigated." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.3" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +}