diff --git a/05_lab_hypothesis_testing_2_challenge_1.ipynb b/05_lab_hypothesis_testing_2_challenge_1.ipynb new file mode 100644 index 0000000..06f3c03 --- /dev/null +++ b/05_lab_hypothesis_testing_2_challenge_1.ipynb @@ -0,0 +1,488 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "4y0j_AeEaXIG" + }, + "source": [ + "# Challenge 1 - T-test\n", + "\n", + "In statistics, t-test is used to test if two data samples have a significant difference between their means. There are two types of t-test:\n", + "\n", + "* **Student's t-test** (a.k.a. independent or uncorrelated t-test). This type of t-test is to compare the samples of **two independent populations** (e.g. test scores of students in two different classes). `scipy` provides the [`ttest_ind`](https://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.stats.ttest_ind.html) method to conduct student's t-test.\n", + "\n", + "* **Paired t-test** (a.k.a. dependent or correlated t-test). This type of t-test is to compare the samples of **the same population** (e.g. scores of different tests of students in the same class). `scipy` provides the [`ttest_re`](https://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.stats.ttest_rel.html) method to conduct paired t-test.\n", + "\n", + "Both types of t-tests return a number which is called the **p-value**. If p-value is below 0.05, we can confidently declare the null-hypothesis is rejected and the difference is significant. If p-value is between 0.05 and 0.1, we may also declare the null-hypothesis is rejected but we are not highly confident. If p-value is above 0.1 we do not reject the null-hypothesis.\n", + "\n", + "Read more about the t-test in [this article](http://b.link/test50) and [this Quora](http://b.link/unpaired97). Make sure you understand when to use which type of t-test." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "id": "g2YMC2cOaXII" + }, + "outputs": [], + "source": [ + "import numpy as np\n", + "import pandas as pd\n", + "import scipy.stats as st" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "PvqrHyGVaXIJ" + }, + "source": [ + "#### Import dataset\n", + "\n", + "In this challenge we will work on the Pokemon dataset. The goal is to test whether different groups of pokemon (e.g. Legendary vs Normal, Generation 1 vs 2, single-type vs dual-type) have different stats (e.g. HP, Attack, Defense, etc.)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "athEywoZaXIJ" + }, + "outputs": [], + "source": [ + "pokemon = pd.read_csv(\"/content/Pokemon.csv\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "wdeEGreaaXIJ" + }, + "source": [ + "#### First we want to define a function with which we can test the means of a feature set of two samples.\n", + "\n", + "In the next cell you'll see the annotations of the Python function that explains what this function does and its arguments and returned value. This type of annotation is called **docstring** which is a convention used among Python developers. The docstring convention allows developers to write consistent tech documentations for their codes so that others can read. It also allows some websites to automatically parse the docstrings and display user-friendly documentations.\n", + "\n", + "Follow the specifications of the docstring and complete the function." + ] + }, + { + "cell_type": "code", + "source": [ + "def t_test_features(s1, s2, features):\n", + "\n", + " \"\"\"Test means of a feature set of two samples\n", + " Args:\n", + " s1 (dataframe): sample 1\n", + " s2 (dataframe): sample 2\n", + " features (list): an array of features to test\n", + "\n", + " Returns:\n", + " dict: a dictionary of t-test scores for each feature where the feature name is the key and the p-value is the value\n", + " \"\"\" # a dictionary of t-test scores for each feature ---> brain melting, but i's easier than it looks like\n", + "\n", + " results = {}\n", + " features = ['HP', 'Attack', 'Defense', 'Sp. Atk', 'Sp. Def', 'Speed', 'Total']\n", + " for feature in features:\n", + " p_value = st.ttest_ind(s1[feature], s2[feature], equal_var = False)[1] # we will assume that it's not equal\n", + " results[feature] = p_value\n", + "\n", + " return results" + ], + "metadata": { + "id": "deegnOQ_gkAV" + }, + "execution_count": 4, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "OkvNL8OpaXIJ" + }, + "source": [ + "#### Using the `t_test_features` function, conduct t-test for Lengendary vs non-Legendary pokemons.\n", + "\n", + "*Hint: your output should look like below:*\n", + "\n", + "```\n", + "{'HP': 1.0026911708035284e-13,\n", + " 'Attack': 2.520372449236646e-16,\n", + " 'Defense': 4.8269984949193316e-11,\n", + " 'Sp. Atk': 1.5514614112239812e-21,\n", + " 'Sp. Def': 2.2949327864052826e-15,\n", + " 'Speed': 1.049016311882451e-18,\n", + " 'Total': 9.357954335957446e-47}\n", + " ```" + ] + }, + { + "cell_type": "code", + "source": [ + "features = ['HP', 'Attack', 'Defense', 'Sp. Atk', 'Sp. Def', 'Speed', 'Total']\n", + "\n", + "# Lengendary vs non-Legendary pokemons\n", + "leg = pokemon[pokemon[\"Legendary\"] == True]\n", + "non_leg = pokemon[pokemon[\"Legendary\"] == False]\n", + "\n", + "t_test_features(leg, non_leg, features)" + ], + "metadata": { + "id": "IevK92C0m0OX" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "jHw-MhNDaXIK" + }, + "source": [ + "#### From the test results above, what conclusion can you make? Do Legendary and non-Legendary pokemons have significantly different stats on each feature?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "WpLfYAyzaXIK" + }, + "outputs": [], + "source": [ + "# H0: There is no significant difference between Legendary and non-Legendary Pokemo,feat wise (like they are equal)\n", + "# H1: There is a significant difference between Legendary and non-Legendary Pokemo,feat wise (like they !=)\n", + "\n", + "result = t_test_features(leg, non_leg, features)\n", + "\n", + "for feat, value in result.items():\n", + " if value > 0.05:\n", + " print(f\"I can not reject the null hypothesis for {feat}\")\n", + " else:\n", + " print(f\"I can reject the null hypothesis for {feat}\")\n", + "\n", + "# if the null is low it has to go. So we can say that they have significantly different stats on each feature - H1" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4JsvA9WpaXIK" + }, + "source": [ + "#### Next, conduct t-test for Generation 1 and Generation 2 pokemons." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "MjLgSas0aXIK" + }, + "outputs": [], + "source": [ + "# H0: There is no significant difference between Generation 1 and Generation 2,feat wise (like they are equal)\n", + "# H1: There is a significant difference between Generation 1 and Generation 2,feat wise (like they !=)\n", + "\n", + "features = ['HP', 'Attack', 'Defense', 'Sp. Atk', 'Sp. Def', 'Speed', 'Total']\n", + "\n", + "# Generation 1 and Generation 2 pokemons\n", + "gen_1 = pokemon[pokemon[\"Generation\"] == 1]\n", + "gen_2 = pokemon[pokemon[\"Generation\"] == 2]\n", + "\n", + "t_test_features(gen_1, gen_2, features)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Nm_LBNWoaXIK" + }, + "source": [ + "#### What conclusions can you make?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "DPKF_yMFaXIL" + }, + "outputs": [], + "source": [ + "# H0: There is no significant difference between Generation 1 and Generation 2,feat wise (like they are equal)\n", + "# H1: There is a significant difference between Generation 1 and Generation 2,feat wise (like they !=) EXCEPT FOR SPEED\n", + "\n", + "result = t_test_features(gen_1, gen_2, features)\n", + "\n", + "for feat, value in result.items():\n", + " if value > 0.05:\n", + " print(f\"I can not reject the null hypothesis for {feat}\")\n", + " else:\n", + " print(f\"I can reject the null hypothesis for {feat}\")\n", + "\n", + "# Gen 2, younger gen have significantly different stats in speed (I totally can relate to that!) and they are probably faster,\n", + "# so i wanna see the stats value now:" + ] + }, + { + "cell_type": "code", + "source": [ + "st.ttest_ind(gen_1[\"Speed\"], gen_2[\"Speed\"], equal_var = False)[0] # yes, not surprised :D" + ], + "metadata": { + "id": "rVN2Ay5btCRS" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "DW-OwoMFaXIL" + }, + "source": [ + "#### Compare pokemons who have single type vs those having two types." + ] + }, + { + "cell_type": "code", + "source": [ + "# H0: There is no significant difference between single type and Two Type Pokemons, feat wise (like they are equal)\n", + "# H1: There is significant difference between single type and Two Type Pokemons, feat wise (like they !=) EXCEPT ----- \"Hit Points\"" + ], + "metadata": { + "id": "zbmy4PEvEnIP" + }, + "execution_count": 13, + "outputs": [] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "id": "NtDYetShaXIL" + }, + "outputs": [], + "source": [ + "# Question: Oh, what did you do on saturday morning?\n", + "\n", + "# single_type ---> are the ones that have \"NaN\" on Type 2\n", + "single_type = pokemon[pokemon[\"Type 2\"].isna()]\n", + "\n", + "# two_types ---> the ones that have a value on Type 2, so droping null will keep only twose with 2 types\n", + "two_types = pokemon.dropna(subset=[\"Type 2\"])\n", + "\n", + "# Answer: This cell (djisus)" + ] + }, + { + "cell_type": "code", + "source": [ + "features = ['HP', 'Attack', 'Defense', 'Sp. Atk', 'Sp. Def', 'Speed', 'Total']\n", + "\n", + "# single_type vs two_types pokemons\n", + "single_type = pokemon[pokemon[\"Type 2\"].isna()]\n", + "two_types = pokemon.dropna(subset=[\"Type 2\"])\n", + "\n", + "result = t_test_features(single_type, two_types, features)\n", + "result" + ], + "metadata": { + "id": "IbHHSxZiD2x0" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "7AZ6nSSJaXIL" + }, + "source": [ + "#### What conclusions can you make?" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "id": "fVy7fO5ZaXIL", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "e9075b0d-49ae-4e8e-9459-4952ec99adb7" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "I can not reject the null hypothesis for HP\n", + "I can reject the null hypothesis for Attack\n", + "I can reject the null hypothesis for Defense\n", + "I can reject the null hypothesis for Sp. Atk\n", + "I can reject the null hypothesis for Sp. Def\n", + "I can reject the null hypothesis for Speed\n", + "I can reject the null hypothesis for Total\n" + ] + } + ], + "source": [ + "# so if the p is low, the null has to go, so we can reject H0 for all features, except HP\n", + "# so pokemons with single type and two type will have similar \"HP\" means\n", + "\n", + "for feat, value in result.items():\n", + " if value > 0.05:\n", + " print(f\"I can not reject the null hypothesis for {feat}\")\n", + " else:\n", + " print(f\"I can reject the null hypothesis for {feat}\")" + ] + }, + { + "cell_type": "code", + "source": [ + "st.ttest_ind(single_type[\"HP\"], two_types[\"HP\"], equal_var = False)" + ], + "metadata": { + "id": "WwbhtabfoUtH" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "1ZyEmpvcaXIL" + }, + "source": [ + "#### Now, we want to compare whether there are significant differences of `Attack` vs `Defense` and `Sp. Atk` vs `Sp. Def` of all pokemons. Please write your code below.\n", + "\n", + "*Hint: are you comparing different populations or the same population?*" + ] + }, + { + "cell_type": "code", + "source": [ + "# H0: There is no significant difference between Attack vs Defense in all pokemons --- > like equal (same sample in both!)\n", + "# H1: There is significant difference between Attack vs Defense in all pokemons -------> like != (same sample in both!)\n", + "# MATCHED PAIR\n", + "\n", + "p_value = st.ttest_rel(pokemon[\"Attack\"], pokemon[\"Defense\"])[1]\n", + "print(p_value)\n", + "\n", + "if p_value > 0.05:\n", + " print(\"I can not reject the null hypothesis\")\n", + "else:\n", + " print(\"We can reject the null hypothesis\")" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "hdSdO98vpUCa", + "outputId": "45c4249e-4513-42e5-b2b3-99c061bb0e2f" + }, + "execution_count": 21, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "1.7140303479358558e-05\n", + "We can reject the null hypothesis\n" + ] + } + ] + }, + { + "cell_type": "code", + "source": [ + "# H0: There is no significant difference between Sp. Atk and Sp. Def in all pokemons --- > like equal (same sample in both!)\n", + "# H1: There is significant difference between Sp. Atk and Sp. Def in all pokemons -------> != (same sample in both!)\n", + "# MATCHED PAIR\n", + "\n", + "p_value = st.ttest_rel(pokemon[\"Sp. Atk\"], pokemon[\"Sp. Def\"])[1]\n", + "print(p_value)\n", + "\n", + "if p_value > 0.05:\n", + " print(\"I can not reject the null hypothesis\")\n", + "else:\n", + " print(\"We can reject the null hypothesis\")" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "P2_L2gbVtksC", + "outputId": "40de9fa9-2c9a-499b-e4fb-1b4e18bffabe" + }, + "execution_count": 22, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "0.3933685997548122\n", + "I can not reject the null hypothesis\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "f-VG2W9XaXIL" + }, + "source": [ + "#### What conclusions can you make?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "1dLMcmCUaXIM" + }, + "outputs": [], + "source": [ + "# There is significant difference between Attack vs Defense in all pokemons, but we cannot say the same for \"special\" attacks and defenses,\n", + "# because they have no significant diferences." + ] + }, + { + "cell_type": "code", + "source": [], + "metadata": { + "id": "3K9i-hBsup6m" + }, + "execution_count": null, + "outputs": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.3" + }, + "colab": { + "provenance": [] + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} \ No newline at end of file diff --git a/05_lab_hypothesis_testing_2_challenge_2.ipynb b/05_lab_hypothesis_testing_2_challenge_2.ipynb new file mode 100644 index 0000000..fe377e0 --- /dev/null +++ b/05_lab_hypothesis_testing_2_challenge_2.ipynb @@ -0,0 +1,598 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "g6903kv52ICO" + }, + "source": [ + "# Challenge 2 - ANOVA\n", + "\n", + "In statistics, **Analysis of Variance (ANOVA)** is also used to analyze the differences among group means. The difference between t-test and ANOVA is the former is ued to compare two groups whereas the latter is used to compare three or more groups. [Read more about the difference between t-test and ANOVA](http://b.link/anova24).\n", + "\n", + "From the ANOVA test, you receive two numbers. The first number is called the **F-value** which indicates whether your null-hypothesis can be rejected. The critical F-value that rejects the null-hypothesis varies according to the number of total subjects and the number of subject groups in your experiment. In [this table](http://b.link/eda14) you can find the critical values of the F distribution. **If you are confused by the massive F-distribution table, don't worry. Skip F-value for now and study it at a later time. In this challenge you only need to look at the p-value.**\n", + "\n", + "The p-value is another number yielded by ANOVA which already takes the number of total subjects and the number of experiment groups into consideration. **Typically if your p-value is less than 0.05, you can declare the null-hypothesis is rejected.**\n", + "\n", + "In this challenge, we want to understand whether there are significant differences among various types of pokemons' `Total` value, i.e. Grass vs Poison vs Fire vs Dragon... There are many types of pokemons which makes it a perfect use case for ANOVA." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "id": "6yg5_qB82ICQ" + }, + "outputs": [], + "source": [ + "import pandas as pd\n", + "import numpy as np\n", + "import scipy.stats as st" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "id": "PbhiNuBS2ICR", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 143 + }, + "outputId": "82a16acf-14c3-47c8-d152-5427ec15d790" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + " # Name Type 1 Type 2 Total HP Attack Defense Sp. Atk Sp. Def \\\n", + "0 1 Bulbasaur Grass Poison 318 45 49 49 65 65 \n", + "1 2 Ivysaur Grass Poison 405 60 62 63 80 80 \n", + "2 3 Venusaur Grass Poison 525 80 82 83 100 100 \n", + "\n", + " Speed Generation Legendary \n", + "0 45 1 False \n", + "1 60 1 False \n", + "2 80 1 False " + ], + "text/html": [ + "\n", + "
| \n", + " | # | \n", + "Name | \n", + "Type 1 | \n", + "Type 2 | \n", + "Total | \n", + "HP | \n", + "Attack | \n", + "Defense | \n", + "Sp. Atk | \n", + "Sp. Def | \n", + "Speed | \n", + "Generation | \n", + "Legendary | \n", + "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", + "1 | \n", + "Bulbasaur | \n", + "Grass | \n", + "Poison | \n", + "318 | \n", + "45 | \n", + "49 | \n", + "49 | \n", + "65 | \n", + "65 | \n", + "45 | \n", + "1 | \n", + "False | \n", + "
| 1 | \n", + "2 | \n", + "Ivysaur | \n", + "Grass | \n", + "Poison | \n", + "405 | \n", + "60 | \n", + "62 | \n", + "63 | \n", + "80 | \n", + "80 | \n", + "60 | \n", + "1 | \n", + "False | \n", + "
| 2 | \n", + "3 | \n", + "Venusaur | \n", + "Grass | \n", + "Poison | \n", + "525 | \n", + "80 | \n", + "82 | \n", + "83 | \n", + "100 | \n", + "100 | \n", + "80 | \n", + "1 | \n", + "False | \n", + "