Data Analysis of NBA Players Challenge.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Analysis of NBA Players\n",
    "\n",
    "This iPython notebook is meant to challenge Georgetown Data Science Certificate students to ensure they have understood or acknowledged the technologies and techniques taught througout the semester. We’ll be analyzing a dataset of NBA players and their performance in the 2013-2014 season. You can download the file [here](http://bit.ly/gtnbads)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Read in a CSV File\n",
    "\n",
    "In the first step, load the dataset into a dataframe. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Load CSV into a dataframe named nba"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remember to import the `pandas` library to get access to Dataframes. Dataframes are two-dimensional arrays (matrices) where each column can be of a different datatype.\n",
    "\n",
    "## Find the number of Players\n",
    "\n",
    "How many players are in the dataset?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Print the number of rows and columns in the dataframe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Look at the First Row of Data\n",
    "\n",
    "What does the first row look like?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Print the first row of data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Find the Average of Each Statistic\n",
    "\n",
    "Find the average value for each statistic. The columns have names like PER (player efficiency rating) and GP (Games Played) that represent the season statistics for each player. For more on the various statistics look [here](http://stats.nba.com/help/glossary)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Print the mean of each column"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Make Pairwise Scatterplots\n",
    "\n",
    "One common way to explore a dataset is to see how different columns correlate to others. We’ll compare the ast, mpg, and to columns. Create a scatter matrix plot of the dataset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%matplotlib inline \n",
    "\n",
    "# Use seaborn or pandas to plot the scatter matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Python, `matplotlib` is the primary plotting package, and `seaborn` is a widely used layer over matplotlib. You could have also used the `pandas` `scatter_matrix` for a similar result. \n",
    "\n",
    "## Make Clusters of the Players \n",
    "\n",
    "One good way to explore this kind of data is to generate cluster plots. These will show which players are most similar. Use Scikit-Learn to cluster the data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Use a clustering model like K-means to cluster the players "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the main Python machine learning package, `scikit-learn`, to fit a `k-means` clustering model and get our cluster labels. In order to cluster properly, we remove any non-numeric columns, or columns with missing values (NA, Nan, etc) with the `get_numeric_data` and `dropna` methods.\n",
    "\n",
    "## Plot Players by Cluster\n",
    "\n",
    "We can now plot out the players by cluster to discover patterns. One way to do this is to first use PCA to make our data 2-dimensional, then plot it, and shade each point according to cluster association."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Use PCA to plot the clusters in 2 dimensions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With Python, we used the PCA class in the scikit-learn library. We used matplotlib to create the plot.\n",
    "\n",
    "## Split into Training and Testing Sets\n",
    "\n",
    "If we want to do supervised machine learning, it’s a good idea to split the data into training and testing sets so we don’t overfit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create train (80%) and test (20%) splits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Python, the recent version of pandas came with a sample method that returns a certain proportion of rows randomly sampled from a source dataframe – this makes the code much more concise. We could also use Scikit-Learns `KFolds` and `train_test_splits` for different types of shuffle and splits in the data set. In both cases, we set a random seed to make the results reproducible.\n",
    "\n",
    "## Univariate Linear Regression \n",
    "\n",
    "Let’s say we want to predict number of assists per player from the turnovers made per player."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Compute the univariate regression of TO to AST"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Scikit-learn has a linear regression model that we can fit and generate predictions from. Note also the use of Lasso and Ridge regressions, though this doesn't apply in the univariate case. \n",
    "\n",
    "## Calculate Summary Statistics for the Model\n",
    "\n",
    "Evaluate the above model using your test set, how well does it perform?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Compute the regression results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we want to get summary statistics about the fit, like r-squared value, we can use the `score` method of the Sckit-Learn model. However, if we want more advanced regression statistics we’ll need to do a bit more. The `statsmodels` package enables many statistical methods to be used in Python and is a good tool to know.\n",
    "\n",
    "## Fit a Random Forest Model \n",
    "\n",
    "Our linear regression worked well in the single variable case, but we suspect there may be nonlinearities in the data. Thus, we want to fit a random forest model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Compute random forest from the predictors \"AGE\", \"MPG\", \"TO\", \"HT\", \"WT\", \"REBR\" to the target, \"AST\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Calculate Error\n",
    "\n",
    "Now that we’ve fit two models, let’s calculate error. We’ll use MSE."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Compute the MSE of the classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The scikit-learn library has a variety of error metrics that we can use\n",
    "\n",
    "## Download a Webpage\n",
    "\n",
    "Now that we have data on NBA players from 2013-2014, let’s scrape some additional data to supplement it. We’ll just look at one box score from the NBA Finals here to save time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Download \"http://www.basketball-reference.com/boxscores/201506140GSW.html\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extract Player Box Scores\n",
    "\n",
    "Now that we have the web page, we’ll need to parse it to extract scores for players."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Use BeautifulSoup to parse the table from the web page"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This will create a list containing two lists, the first with the box score for CLE, and the second with the box score for GSW. Both contain the headers, along with each player and their in-game stats. We won’t turn this into more training data now, but it could easily be transformed into a format that could be added to our nba dataframe.\n",
    "\n",
    "BeautifulSoup, the most commonly used web scraping package. It enables us to loop through the tags and construct a list of lists in a straightforward way."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}