Skip to content

Lab11 #74

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jdcos wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Lab11 #74

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,569 changes: 1,569 additions & 0 deletions your-code/.ipynb_checkpoints/challenge-1-checkpoint.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

195 changes: 195 additions & 0 deletions your-code/.ipynb_checkpoints/challenge-2-checkpoint.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Challenge 2\n",
"\n",
"In this challenge we will continue working with the `Pokemon` dataset. We will attempt solving a slightly more complex problem in which we will practice the iterative data analysis process you leaned in [this video](https://www.youtube.com/watch?v=xOomNicqbkk).\n",
"\n",
"The problem statement is as follows:\n",
"\n",
"**You are at a Pokemon black market planning to buy a Pokemon for battle. All Pokemon are sold at the same price and you can only afford to buy one. You cannot choose which specific Pokemon to buy. However, you can specify the type of the Pokemon - one type that exists in either `Type 1` or `Type 2`. Which type should you choose in order to maximize your chance of receiving a good Pokemon?**\n",
"\n",
"To remind you about the 3 steps of iterative data analysis, they are:\n",
"\n",
"1. Setting Expectations\n",
"1. Collecting Information\n",
"1. Reacting to Data / Revising Expectations\n",
"\n",
"Following the iterative process, we'll guide you in completing the challenge."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Problem Solving Iteration 1\n",
"\n",
"In this iteration we'll analyze the problem and identify the breakthrough. The original question statement is kind of vague because we don't know what a *good pokemon* really means as represented in the data. We'll start by understanding the dataset and see if we can find some insights."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import libraries\n",
"import numpy as np\n",
"import pandas as pd"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"# Importing the dataset"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"From the data it seems whether a pokemon is good depends on its abilities as represented in the fields of `HP`, `Attack`, `Defense`, `Sp. Atk`, `Sp. Def`, `Speed`, and `Total`. We are not sure about `Generation` and `Legendary` because they are not necessarily the decisive factors of the pokemon abilities.\n",
"\n",
"But `HP`, `Attack`, `Defense`, `Sp. Atk`, `Sp. Def`, `Speed`, and `Total` are a lot of fields! If we look at them all at once it's very complicated. This isn't Mission Impossible but it's ideal that we tackle this kind of problem after we learn Machine Learning (which you will do in Module 3). For now, is there a way to consolidate the fields we need to look into?\n",
"\n",
"Fortunately there seems to be a way. It appears the `Total` field is computed based on the other 6 fields. But we need to prove our theory. If we can approve there is a formula to compute `Total` based on the other 6 abilities, we only need to look into `Total`.\n",
"\n",
"We have the following expectation now:\n",
"\n",
"#### The `Total` field is computed based on `HP`, `Attack`, `Defense`, `Sp. Atk`, `Sp. Def`, and `Speed`.\n",
"\n",
"We need to collect the following information:\n",
"\n",
"* **What is the formula to compute `Total`?**\n",
"* **Does the formula work for all pokemon?**\n",
"\n",
"In the cell below, make a hypothesis on how `Total` is computed and test your hypothesis."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# your code here"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Problem Solving Iteration 2\n",
"\n",
"Now that we have consolidated the abilities fields, we can update the problem statement. The new problem statement is:\n",
"\n",
"### Which pokemon type is most likely to have the highest `Total` value?\n",
"\n",
"In the updated problem statement, we assume there is a certain relationship between the `Total` and the pokemon type. But we have two *type* fields (`Type 1` and `Type 2`) that have string values. In data analysis, string fields have to be transformed to numerical format in order to be analyzed. \n",
"\n",
"In addition, keep in mind that `Type 1` always has a value but `Type 2` is sometimes empty (having the `NaN` value). Also, the pokemon type we choose may be either in `Type 1` or `Type 2`.\n",
"\n",
"Now our expectation is:\n",
"\n",
"#### `Type 1` and `Type 2` string variables need to be converted to numerical variables in order to identify the relationship between `Total` and the pokemon type.\n",
"\n",
"The information we need to collect is:\n",
"\n",
"#### How to convert two string variables to numerical?\n",
"\n",
"Let's address the first question first. You can use a method called **One Hot Encoding** which is frequently used in machine learning to encode categorical string variables to numerical. The idea is to gather all the possible string values in a categorical field and create a numerical field for each unique string value. Each of those numerical fields uses `1` and `0` to indicate whether the data record has the corresponding categorical value. A detailed explanation of One Hot Encoding can be found in [this article](https://hackernoon.com/what-is-one-hot-encoding-why-and-when-do-you-have-to-use-it-e3c6186d008f). You will formally learn it in Module 3.\n",
"\n",
"For instance, if a pokemon has `Type 1` as `Poison` and `Type 2` as `Fire`, then its `Poison` and `Fire` fields are `1` whereas all other fields are `0`. If a pokemon has `Type 1` as `Water` and `Type 2` as `NaN`, then its `Water` field is `1` whereas all other fields are `0`.\n",
"\n",
"#### In the next cell, use One Hot Encoding to encode `Type 1` and `Type 2`. Use the pokemon type values as the names of the numerical fields you create.\n",
"\n",
"The new numerical variables you create should look like below:\n",
"\n",
"![One Hot Encoding](../images/one-hot-encoding.png)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# your code here"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Problem Solving Iteration 3\n",
"\n",
"Now we have encoded the pokemon types, we will identify the relationship between `Total` and the encoded fields. Our expectation is:\n",
"\n",
"#### There are relationships between `Total` and the encoded pokemon type variables and we need to identify the correlations.\n",
"\n",
"The information we need to collect is:\n",
"\n",
"#### How to identify the relationship between `Total` and the encoded pokemon type fields?\n",
"\n",
"There are multiple ways to answer this question. The easiest way is to use correlation. In the cell below, calculate the correlation of `Total` to each of the encoded fields. Rank the correlations and identify the #1 pokemon type that is most likely to have the highest `Total`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# your code here"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Bonus Question\n",
"\n",
"Say now you can choose both `Type 1` and `Type 2` of the pokemon. In order to receive the best pokemon, which types will you choose?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# your code here"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.9"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
147 changes: 147 additions & 0 deletions your-code/.ipynb_checkpoints/challenge-3-checkpoint.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Challenge 3\n",
"\n",
"In this challenge we will work on the `Orders` data set. In your work you will apply the thinking process and workflow we showed you in Challenge 2.\n",
"\n",
"You are serving as a Business Intelligence Analyst at the headquarter of an international fashion goods chain store. Your boss today asked you to do two things for her:\n",
"\n",
"**First, identify two groups of customers from the data set.** The first group is **VIP Customers** whose **aggregated expenses** at your global chain stores are **above the 95th percentile** (aka. 0.95 quantile). The second group is **Preferred Customers** whose **aggregated expenses** are **between the 75th and 95th percentile**.\n",
"\n",
"**Second, identify which country has the most of your VIP customers, and which country has the most of your VIP+Preferred Customers combined.**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Q1: How to identify VIP & Preferred Customers?\n",
"\n",
"We start by importing all the required libraries:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# import required libraries\n",
"import numpy as np\n",
"import pandas as pd"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, extract and import `Orders` dataset into a dataframe variable called `orders`. Print the head of `orders` to overview the data:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# your code here"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"\n",
"\"Identify VIP and Preferred Customers\" is the non-technical goal of your boss. You need to translate that goal into technical languages that data analysts use:\n",
"\n",
"## How to label customers whose aggregated `amount_spent` is in a given quantile range?\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We break down the main problem into several sub problems:\n",
"\n",
"#### Sub Problem 1: How to aggregate the `amount_spent` for unique customers?\n",
"\n",
"#### Sub Problem 2: How to select customers whose aggregated `amount_spent` is in a given quantile range?\n",
"\n",
"#### Sub Problem 3: How to label selected customers as \"VIP\" or \"Preferred\"?\n",
"\n",
"*Note: If you want to break down the main problem in a different way, please feel free to revise the sub problems above.*\n",
"\n",
"Now in the workspace below, tackle each of the sub problems using the iterative problem solving workflow. Insert cells as necessary to write your codes and explain your steps."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# your code here"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we'll leave it to you to solve Q2 & Q3, which you can leverage from your solution for Q1:\n",
"\n",
"## Q2: How to identify which country has the most VIP Customers?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# your code here"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Q3: How to identify which country has the most VIP+Preferred Customers combined?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# your code here"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.9"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Loading