IdentifyingIrisPlantsWithMachineLearning.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/damonLL/amplifyapp/blob/main/IdentifyingIrisPlantsWithMachineLearning.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_9QN9KnoTJC3"
      },
      "source": [
        "# Identifying Iris Plants with Machine Learning\n",
        "\n",
        "In this lab, we are going to replicate a famous experiment performed by Sir Ronald Fisher in his 1936 paper *The use of multiple measurements in taxonomic problems*. Why are we repeating a nearly 100-year-old experiment? Because now we can let a machine learn how to do it instead of doing the math ourselves. Yeah!\n",
        "\n",
        "## First, you need to make a copy of this notebook for yourself.\n",
        "In the top menu, choose File -> Save a Copy in Drive. Save a copy for yourself in your own Google Drive for later reference! Continue with the lab on your copy.\n",
        "\n",
        "### I'm excited! What are we going to do?\n",
        "I'm glad you asked. Fisher's dataset contains data related to 3 different species of Iris plants: *Iris setosa*, *Iris versicolor*, and *Iris virginica*, with 50 sets of data points for each species. Each data point consists of 4 features:\n",
        " * Sepal length\n",
        " * Sepal width\n",
        " * Petal length\n",
        " * Petal width\n",
        "\n",
        "All of these features are measured in centimeters. Petals you're probably familiar with, but sepals are the green leaf-like structures directly under the petals that support them. By taking these 4 measurements, he was able to statistically (and reliably!) predict the species from measurements of an unknown flower. We're going to do the same thing, but use machine learning to make the prediction for us.\n",
        "\n",
        "## 1. Load the Data\n",
        "\n",
        "First, we have to do our standard imports. We'll also check which version of TensorFlow we're using - any version >= 2.3.0 should work. This loads the libraries we'll need for the rest of the exercise. Run the cell by pressing the Play button on the left side, or pressing Control + Enter on your keyboard.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kIIhIPMRk8JH"
      },
      "source": [
        "import tensorflow as tf\n",
        "import tensorflow_datasets as tfds\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "from matplotlib import pyplot as plt\n",
        "import os\n",
        "\n",
        "print(tf.__version__)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MSo-rgrSTHut"
      },
      "source": [
        "The Iris dataset is so well known that TensorFlow has an automatic way to import the original dataset, which we will use to train our model. This cell also downloads a set of 30 iris data examples that we can use to test our model once it is trained.\n",
        "\n",
        "We will be using two libraries to help us work with the data: NumPy and Pandas. The only thing you need to know for the moment is that Pandas uses a structure called a DataFrame, commonly abbreviated as `df`, to hold the data. These two libraries will be important as you continue your machine learning journey."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ytd2xhSLlN2-"
      },
      "source": [
        "#Training Data - Using TensorFlow to automatically load\n",
        "iris_features = [\"sepal_length\", \"sepal_width\", \"petal_length\",\"petal_width\"]\n",
        "iris_labels = [\"Iris setosa\", \"Iris versicolor\", \"Iris virginica\"]\n",
        "\n",
        "training_features, training_labels = tfds.as_numpy(tfds.load('iris', split='train', as_supervised=True, batch_size=-1))\n",
        "iris_df = pd.DataFrame(data=training_features, columns=iris_features)\n",
        "iris_df['species'] = [iris_labels[x] for x in training_labels]\n",
        "\n",
        "#Testing Data - Downloading the test data, then loading it\n",
        "test_col_names = iris_features + ['species']\n",
        "test_data_url = \"https://raw.githubusercontent.com/selva86/datasets/master/iris_test.csv\"\n",
        "test_file = tf.keras.utils.get_file(fname=os.path.basename(test_data_url), origin=test_data_url)\n",
        "\n",
        "test_df = pd.read_csv(test_file, delimiter=',', skiprows=1, header=None, names=test_col_names)\n",
        "\n",
        "test_features = test_df.iloc[:,0:4]\n",
        "test_labels = test_df.iloc[:,-1]\n",
        "test_df['species'] = [iris_labels[x] for x in test_labels]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mXq1gK7a-iN2"
      },
      "source": [
        "## 2. Visualize the Data\n",
        "\n",
        "Let's start with a simple count. We expect to have 150 sets of features in the training set, as well as 150 labels. We'll print the length of the lists containing the features and labels for the training and test sets."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oqd6EEJtnkaf"
      },
      "source": [
        "print(f\"Training data. Features: {len(training_features)}. Labels: {len(training_labels)}.\")\n",
        "print(f\"Testing data.  Features: {len(test_features)}. Labels: {len(test_labels)}.\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DPkwReclBWdl"
      },
      "source": [
        "That was a lot of code to show us 4 numbers. Let's let Pandas do the heavy lifting. We'll examine statistics about the training and test data calculated automatically for us by the DataFrame.\n",
        "\n",
        "Run the next two cells and take a look at the statistics."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mgp0EAKkXWd8"
      },
      "source": [
        "iris_df.describe()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oyijDB2bCpJh"
      },
      "source": [
        "test_df.describe()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X3aO2FtVCv9L"
      },
      "source": [
        "Now that we've seen the big picture, let's look at some individual data points. We'll look at the top 15 examples in the training and test data. For context, remember that these data points are measurements in centimeters.\n",
        "\n",
        "Run the following two cells."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nQ0DhuxxARUQ"
      },
      "source": [
        "iris_df.head(15)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AeTs3bAXEPjq"
      },
      "source": [
        "test_df.head(15)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nPJ1dO6DExDM"
      },
      "source": [
        "### Wait. There is another column in this view.\n",
        "\n",
        "Yes, there is. How do you expect to calculate mean and standard deviation on plain text? The DataFrame was smart enough to not include this column in the ```describe``` statistics.\n",
        "\n",
        "Looking at the individual data points allows us to see the species, which is our label. I've cleaned this up a bit for you to make it easier to read. In the raw data, the species is a 0, 1, or 2. It's much nicer to look at the actual species name.\n",
        "\n",
        "This still looks like alphanumeric soup, though. There are another 135 rows of data we didn't even look at. We need a better way to visualize the data. Pandas to the rescue again! We'll create a scatter matrix, which graphs each feature against every other feature. This works well because all of the features have the same unit of measurement, and we only have four features, so the graphs aren't out of control. Along the diagonal where a feature would graph against itself is instead a histogram showing the distribution of that feature. Super useful!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DSoYwwxQY0l-"
      },
      "source": [
        "colormap = {'Iris setosa':'orange', 'Iris versicolor':'teal', 'Iris virginica':'purple'}\n",
        "colors = iris_df['species'].map(lambda x: colormap[x])\n",
        "scattermatrix = pd.plotting.scatter_matrix(iris_df, color=colors, figsize=(20,15))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cv9VAmu-Nys1"
      },
      "source": [
        "We can visualize this one more way on a RadViz plot. This shows us how separable the data is by the different features."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0VQ-NlJYefmA"
      },
      "source": [
        "plt.figure(figsize=(20,15))\n",
        "radviz = pd.plotting.radviz(iris_df, 'species', color=colors)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9IJH3lVsO42t"
      },
      "source": [
        "We can see that the *Iris setosa*, shown in orange, is pretty easy to separate from the other two. You can draw a line through every one of those graphs to separate it from the other two labels, also called classes. Unfortunately, telling *Iris versicolor* from *Iris virginica* is more difficult. We can't definitively separate those two classes with a simple line because the examples are very close together, and even overlap substantially. Lucky for us, we have a way to let the machine learn the differences so we don't have to!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IGB3ug1PPFQK"
      },
      "source": [
        "## 3. Teach the Machine about Irises\n",
        "\n",
        "TensorFlow allows us to have as much control over the machine learning process as we need. Since we're working on a pretty small problem that we don't need to fiddle with too much, we're going to work with TensorFlow using a high-level framework called Keras that allows us to get right to work without having to tweak a bunch of settings. As you go about your own machine learning projects, it's usually a good idea to try the simplest method first. If nothing else, it gives you a good baseline to compare more complex methods against.\n",
        "\n",
        "A lot of trial and error goes into finding a good architecture for understanding your individual data. This data, however, has been solved many times before, so architectures that produce decent results are widely known. We're going to use a fairly simple Neural Network. The input shape is `(4,)` because we're passing in 4 features (petal and sepal length and width). The hidden layers are densely connected.  The last layer is our output, and it requires 3 neurons, one for each of the three classes we are training the machine to identify. The output will be logits, which is an individual probability metric for how likely the machine thinks the features belong to each class.\n",
        "\n",
        "We also have to tell it how to correct itself (the `adam` optimizer), what it should use to judge correctness (Sparse Categorical Crossentropy) and any other metrics we'd like it to report (Sparse Categorical Accuracy: How much of the  time was the class prediction correct?).\n",
        "\n",
        "### Woah, hoss. Back up. That was a lot of fancy words.\n",
        "You're right, it was. I hope your eyes didn't glaze over too badly. I wanted to introduce you to what we were doing so you get some exposure to the terminology, but fear not! I have a shorter summary that will probably make more sense.\n",
        "\n",
        "We're going to use a structure that mimics the way the human brain works to teach the machine about irises. We have to tell it how many features to expect for each example (4), how many combinations of information it can learn (total number of connections, which is 4 (our input) x 10 x 10 x 3), how complex we expect the data to be to learn (mostly related to how many hidden layers we have, 2), and finally, what we expect the output to be (a probability score).\n",
        "\n",
        "\n",
        "### You said that was going to be simpler.\n",
        "Fair enough. That was still pretty complex. Don't worry too much about the finer points right now. This is your first model. That's why I'm giving you all the code you need so you can build it today and study it later."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hDJL8CZNGBQh"
      },
      "source": [
        "iris_model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(10, activation=tf.nn.relu, input_shape=(4,)),\n",
        "    tf.keras.layers.Dense(10, activation=tf.nn.relu),\n",
        "    tf.keras.layers.Dense(3)\n",
        "  ])\n",
        "\n",
        "iris_model.compile(optimizer='adam', loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=[tf.keras.metrics.SparseCategoricalAccuracy()])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RDipYrfYW4nn"
      },
      "source": [
        "Ok, so we built our structure, but it isn't trained yet. We haven't actually shown it any data. That's what this next step will do. We'll use the ```fit``` method to train our architecture what the data should look like. We will show it 16 examples (rows) at a time and have it update its weights after seeing those. We'll do this for 100 epochs, which means we will show it the full training dataset 100 times. Have you ever studied something multiple times to understand it better? That's exactly what we're having the machine do."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JftkoI2qON8U"
      },
      "source": [
        "iris_model.fit(training_features, training_labels, verbose=2, batch_size=16, epochs=100)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wW0sf6RFZwfg"
      },
      "source": [
        "My `sparse_categorical_accuracy` came out to 0.9733, which means it predicted correctly 97.33% of the time on the training data. That's not bad for only having 150 data points! How did you do? Your value should be close to that, but might not be exactly the same.\n",
        "\n",
        "Let's take a look at the predictions. We're going to define a short function to make our predictions, then format them for easier viewing. The actual predictions are being made when we use `iris_model.predict`. Everything after that is just some clever output formatting."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uWV-PX-qT2qi"
      },
      "source": [
        "def predict(features, labels):\n",
        "    predictions = iris_model.predict(features)\n",
        "\n",
        "    pred_df = pd.DataFrame(columns=['Actual','Prediction'])\n",
        "    pred_df['Actual'] = [iris_labels[x] for x in labels]\n",
        "    pred_df['Prediction'] = [iris_labels[x] for x in [np.argmax(p) for p in predictions]]\n",
        "    pred_df['Correct'] = pred_df['Actual'] == pred_df['Prediction']\n",
        "\n",
        "    return pred_df"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wU6TQ9zfaIDt"
      },
      "source": [
        "Normally, you don't try to predict using the data you used for training, but since we have limited data, we're going to do it just to see what the output should look like. We'll see the `Actual` value, which is the true species of the flower. Beside that, we'll see what `Prediction` for the species that the model made, and finally if the prediction was `Correct`. That's much easier to read than comparing the values ourselves. We'll look at the first 30 rows."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_JvmltyPM034"
      },
      "source": [
        "training_predictions = predict(training_features[:30], training_labels[:30])\n",
        "training_predictions"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4Pnz4xeZbvR7"
      },
      "source": [
        "It shows exactly what we expected from the accuracy percentage: it's nearly always right.\n",
        "\n",
        "But now, how do we get the same accuracy score for our test data? We have another function exactly for this: `evaluate`! We can pass it our test features and test labels, and it will calculate the same metrics it did as the model was training."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2zOs_nClLCnx"
      },
      "source": [
        "iris_model.evaluate(test_features, test_labels)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m6zyq8BEdqR2"
      },
      "source": [
        "My accuracy is 0.9667, or 96.67%. That's about half a percentage point difference from the training accuracy, which means this model is well trained.\n",
        "\n",
        "Finally, let's use our fancy formatting function to take a look at the test data set predictions."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fv6IvWMNFi5M"
      },
      "source": [
        "test_predictions = predict(test_features, test_labels)\n",
        "test_predictions"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-QTHx8S9drO0"
      },
      "source": [
        "## Congratulations on building your first machine learning model!\n",
        "Compile, Fit, Evaluate, Predict. It's just that easy. You're off to a great start on your machine learning journey!"
      ]
    }
  ]
}