00_tensorflow_fundamentals.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyPMb0vPzMu40vmHTubp88HC",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/SaketMunda/tensorflow-fundamentals/blob/master/00_tensorflow_fundamentals.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hKsu1pA5GSV_",
        "outputId": "6297c5ab-7ff5-4497-a83f-36ceac7999c2"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2.9.2\n"
          ]
        }
      ],
      "source": [
        "# Import TensorFlow\n",
        "import tensorflow as tf\n",
        "print(tf.__version__) # find the version number"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## What are Tensors ?\n",
        "\n",
        "You can think of tensors as a multi-dimensional numercial representation (also referred to as n-dimensional, where n can be any number) of something. Where something can be almost anything you can imagine:\n",
        "\n",
        "* It could be numbers themselves (using tensors to represent the price of houses).\n",
        "* It could be an image (using tensors to represent the pixels of an image).\n",
        "* It could be text (using tensors to represent words).\n",
        "* Or it could be some other form of information (or data) you want to represent with numbers.\n",
        "\n",
        "The main difference between tensors and NumPy arrays (also an n-dimensional array of numbers) is that tensors can be used on GPUs and TPUs.\n",
        "\n",
        "The benefit of being able to run on GPUs and TPUs is faster computation, this means, if we wanted to find patterns in the numerical representation of our data, we can generally find them faster using GPUs and TPUs."
      ],
      "metadata": {
        "id": "1PT-DlywHxYW"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Creating Tensors with `tf.constant()`"
      ],
      "metadata": {
        "id": "PlV3IoufJyrG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# create a scalar (rank 0 tensors)\n",
        "scalar = tf.constant(7)\n",
        "scalar"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "l3abXnFtJ9sa",
        "outputId": "5e14951a-7cc0-43e4-e9ad-53486282a216"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=7>"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "here notice that shape is empty, it means it has no dimensions, it's just a number"
      ],
      "metadata": {
        "id": "A3HV3-2gKQE5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# check the number of dimensions of a tensor\n",
        "scalar.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Tq9INbhEKa5S",
        "outputId": "cc599631-286f-4fce-e364-24cbeae70d31"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# create a vector (more than 0 dimensions)\n",
        "vector = tf.constant([2,7])\n",
        "vector"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "n7E3jRBPKdpx",
        "outputId": "980c79ca-0fcf-4d42-d1a6-14594e44f5a2"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([2, 7], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# check the dimension \n",
        "vector.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TaxfUaL6Ks-A",
        "outputId": "49f2b413-27f0-4d8a-9978-7c2142d5f03a"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "it says, it has 1 dimension"
      ],
      "metadata": {
        "id": "Ht0MVKX-KxEx"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# let's create a matrix\n",
        "matrix = tf.constant([[1,2],\n",
        "                      [3,4]])\n",
        "matrix"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2RS7jtELK3mz",
        "outputId": "8f235e2f-093d-4385-f188-904cd81c7ec6"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[1, 2],\n",
              "       [3, 4]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "check the value of shape this time, it has 2 numbers which means this is a 2 dimensional tensor constant"
      ],
      "metadata": {
        "id": "2Ri8E7fUOzqm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# checking the ndim\n",
        "matrix.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "l_8NoibaPAMR",
        "outputId": "a399b8f9-3248-43e8-8b89-7485bb94f0e2"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "by default if we don't pass any dtype value while creating constant tensor it'll create int32 or float32 tensors\n",
        "\n",
        "This is also called 32-bit precision(the higher the number, the more precise the number, the more space it takes up on your computer)."
      ],
      "metadata": {
        "id": "VoMO3Pr2PDBu"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# create another matrix and define the datatype\n",
        "another_matrix = tf.constant([[1.,2.],\n",
        "                              [2.,3.],\n",
        "                              [4.,5.]], dtype=tf.float16)\n",
        "another_matrix"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "S-gNltHSPmMC",
        "outputId": "5a3b1ce0-c401-4f72-ed87-2026390c064f"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float16, numpy=\n",
              "array([[1., 2.],\n",
              "       [2., 3.],\n",
              "       [4., 5.]], dtype=float16)>"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# even though another_matrix contains the more numbers, its dimensions stay the same\n",
        "another_matrix.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2uR8tAnnQHIi",
        "outputId": "b7495a22-f73f-4313-94f9-a14c048189d5"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# How about a tensor ? (more than 2 dimensions, although, all of the above items are also technically tensors)\n",
        "tensor = tf.constant([[[1,2,3],\n",
        "                       [4,5,6]],\n",
        "                      [[7,8,9],\n",
        "                       [10,11,12]],\n",
        "                      [[13,14,15],\n",
        "                       [16,17,18]]])\n",
        "tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dvRUjmd0QUR5",
        "outputId": "9983a9bc-5028-4ec9-b3b4-5a3aafbdd5ca"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2, 3), dtype=int32, numpy=\n",
              "array([[[ 1,  2,  3],\n",
              "        [ 4,  5,  6]],\n",
              "\n",
              "       [[ 7,  8,  9],\n",
              "        [10, 11, 12]],\n",
              "\n",
              "       [[13, 14, 15],\n",
              "        [16, 17, 18]]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tensor.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "M0ktRru7RWgI",
        "outputId": "a15ee254-b13a-4f98-c5c8-da4dd3696588"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Creating tensors with `tf.Variable()`\n",
        "\n",
        "You can also (although you likely rarely will, because often, when working with data, tensors are created for you automatically) create tensors using `tf.Variable()`\n",
        "\n",
        "The difference between `tf.constant()` and `tf.Variable()` is tensors with `tf.constant()` are immutable (can't be changed, can only be used to create a new tensor), where as, tensors created with `tf.Variable()` are mutable (can be changed)."
      ],
      "metadata": {
        "id": "YXmtOfpUR-AP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# create the same tensor with tf.Variable() and tf.constant()\n",
        "changeable_tensor = tf.Variable([10,7])\n",
        "unchangeable_tensor = tf.constant([10,7])\n",
        "\n",
        "changeable_tensor, unchangeable_tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fKwMuAyZTBJy",
        "outputId": "cc61387b-52fc-49b4-ae6d-8198d24add32"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Variable 'Variable:0' shape=(2,) dtype=int32, numpy=array([10,  7], dtype=int32)>,\n",
              " <tf.Tensor: shape=(2,), dtype=int32, numpy=array([10,  7], dtype=int32)>)"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now let's try to change one of the elements of the changeable tensor"
      ],
      "metadata": {
        "id": "GcUu_fV4UMX_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "changeable_tensor[0] = 7\n",
        "changeable_tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 184
        },
        "id": "40yJDngNUYph",
        "outputId": "bc15e5d6-8037-41ed-edae-4a38180fb1b5"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "error",
          "ename": "TypeError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-14-05d0bdd98eb4>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mchangeable_tensor\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m7\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      2\u001b[0m \u001b[0mchangeable_tensor\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mTypeError\u001b[0m: 'ResourceVariable' object does not support item assignment"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "To change an element of a `tf.Variable()` tensor requires the `assign()` method"
      ],
      "metadata": {
        "id": "VFISHyBKV00r"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "changeable_tensor[0].assign(7)\n",
        "changeable_tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "K-9LBKXEWNCM",
        "outputId": "e2f31ced-fdd4-4458-8229-ae1240168233"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'Variable:0' shape=(2,) dtype=int32, numpy=array([7, 7], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now let's try to change a value in unchangeable tensor i.e `tf.constant()`"
      ],
      "metadata": {
        "id": "iEi-TGwsWUB6"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "unchangeable_tensor[0].assign(7)\n",
        "unchangeable_tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 305
        },
        "id": "SNAO1qO7WWFT",
        "outputId": "2c342b76-82ef-4803-c3fc-2b93777674a7"
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "error",
          "ename": "AttributeError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-16-958e786d8d1f>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0munchangeable_tensor\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0massign\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m7\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      2\u001b[0m \u001b[0munchangeable_tensor\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py\u001b[0m in \u001b[0;36m__getattr__\u001b[0;34m(self, name)\u001b[0m\n\u001b[1;32m    444\u001b[0m         \u001b[0mnp_config\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0menable_numpy_behavior\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    445\u001b[0m       \"\"\")\n\u001b[0;32m--> 446\u001b[0;31m     \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__getattribute__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    447\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    448\u001b[0m   \u001b[0;34m@\u001b[0m\u001b[0mstaticmethod\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mAttributeError\u001b[0m: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'assign'"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Creating random tensors\n",
        "\n",
        "Random tensors are tensors of some arbitrary size which contain random numbers.\n",
        "\n",
        "Why would you want to create random tensors ?\n",
        "\n",
        "This is what neural networks use to initialize their weights (patterns) that they're trying to learn in the data."
      ],
      "metadata": {
        "id": "iYWcf3TkWv60"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can create random tensors by using the `tf.random.Generator` class"
      ],
      "metadata": {
        "id": "IffYDRHjXT_E"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# create two random (but the same) tensors\n",
        "random_1 = tf.random.Generator.from_seed(42) # set the seed for reproducibility\n",
        "random_1 = random_1.normal(shape=(3,2)) # create tensor from a normal distribution\n",
        "random_2 = tf.random.Generator.from_seed(42)\n",
        "random_2 = random_2.normal(shape=(3,2))\n",
        "\n",
        "random_1, random_2, random_1 == random_2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SFml7eAXYu_c",
        "outputId": "a1850c2d-bd60-4000-8a13-b318b0b3924e"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              " array([[-0.7565803 , -0.06854702],\n",
              "        [ 0.07595026, -1.2573844 ],\n",
              "        [-0.23193763, -1.8107855 ]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              " array([[-0.7565803 , -0.06854702],\n",
              "        [ 0.07595026, -1.2573844 ],\n",
              "        [-0.23193763, -1.8107855 ]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=bool, numpy=\n",
              " array([[ True,  True],\n",
              "        [ True,  True],\n",
              "        [ True,  True]])>)"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The random tensors we've made are actually pseudorandom numbers (they appear as random, but really aren't)"
      ],
      "metadata": {
        "id": "dgHC_g20ZPIJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# create two random (and different) tensors\n",
        "random_3 = tf.random.Generator.from_seed(42)\n",
        "random_3 = random_3.normal(shape=(3,2))\n",
        "random_4 = tf.random.Generator.from_seed(7)\n",
        "random_4 = random_4.normal(shape=(3,2))\n",
        "\n",
        "# check the tensors and see if they are equal\n",
        "random_3, random_4, random_3 == random_4, random_1==random_3"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "U3UjI2pvZu5z",
        "outputId": "10e9b5c9-d48c-4826-9f76-aae93670024c"
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              " array([[-0.7565803 , -0.06854702],\n",
              "        [ 0.07595026, -1.2573844 ],\n",
              "        [-0.23193763, -1.8107855 ]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              " array([[-1.3240396 ,  0.28785667],\n",
              "        [-0.8757901 , -0.08857018],\n",
              "        [ 0.69211644,  0.84215707]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=bool, numpy=\n",
              " array([[False, False],\n",
              "        [False, False],\n",
              "        [False, False]])>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=bool, numpy=\n",
              " array([[ True,  True],\n",
              "        [ True,  True],\n",
              "        [ True,  True]])>)"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Shuffle the order of a Tensor\n",
        "\n",
        "Why we want to shuffle the order of a tensor ?\n",
        "\n",
        "Let's say you working with 15K images of cats and dogs and the first 10K images are of cats and the next 5K were of dogs. This order could effect how a neural network leanrs (it may overfit by learning the order of the data), instead, it might be a good idea to move your data around."
      ],
      "metadata": {
        "id": "eKZCFA-0aXY0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Shuffle a tensor (valuable for when you want to shuffle your data)\n",
        "not_shuffled = tf.constant([[10,7],\n",
        "                            [3,4],\n",
        "                            [2,5]])\n",
        "\n",
        "# Gets different results each time\n",
        "tf.random.shuffle(not_shuffled)"
      ],
      "metadata": {
        "id": "JMoDd6eRaebb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a475c02b-43df-481a-8ca2-ce2ba912be52"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[ 3,  4],\n",
              "       [ 2,  5],\n",
              "       [10,  7]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Shuffle in the same order every time using the same parameter (won't actually be the same)\n",
        "tf.random.shuffle(not_shuffled, seed=42)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "P9sYiYwKVTJr",
        "outputId": "a4ded503-ecdb-4e4a-f529-01af945c2e02"
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[ 2,  5],\n",
              "       [ 3,  4],\n",
              "       [10,  7]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "But even if we set the random seed value, everytime when we run it returns different order. Why ?\n",
        "\n",
        "It's due to rule #4 of the [`tf.random.set_seed()`](https://www.tensorflow.org/api_docs/python/tf/random/set_seed) documentation\n",
        "\n",
        "> Rule 4: If the global and the operation seed are set: Both seeds are used in conjuction to determine the random sequence.\n",
        "\n",
        "Because, \"Operations that rely on a random seed actually derive it from two seeds: the global and operation-level seeds. This sets the global seed.\""
      ],
      "metadata": {
        "id": "9XG5PBFWVlC_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Shuffle in the same order every time\n",
        "\n",
        "# set the global seed\n",
        "tf.random.set_seed(42)\n",
        "\n",
        "# set the operational seed\n",
        "tf.random.shuffle(not_shuffled, seed=42)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qbgZ-4_OWu74",
        "outputId": "78922d2a-7bcb-4929-f563-e40ab3ca52a9"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 3,  4],\n",
              "       [ 2,  5]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# set the global random seed\n",
        "tf.random.set_seed(42)\n",
        "\n",
        "tf.random.shuffle(not_shuffled)\n",
        "\n",
        "# this way also it can reproduce same results"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iVuvslGQW8Ih",
        "outputId": "1bd9089d-5c5a-4841-9979-c0d5a63610aa"
      },
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[ 3,  4],\n",
              "       [ 2,  5],\n",
              "       [10,  7]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Other ways to make tensors"
      ],
      "metadata": {
        "id": "saemF5rHXM85"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Make a tensor of all ones\n",
        "tf.ones([10,7])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cosUIv-KXcx_",
        "outputId": "24980baf-6915-4a26-95f5-12a11f7a3aaf"
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(10, 7), dtype=float32, numpy=\n",
              "array([[1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Make a tensor of all zeroes\n",
        "tf.zeros([10,7])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1jyg2RGLXg_G",
        "outputId": "2544791f-8e0a-4b1c-fb51-20ed36f56158"
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(10, 7), dtype=float32, numpy=\n",
              "array([[0., 0., 0., 0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0., 0., 0., 0.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Creating Tensors from Numpy arrays\n",
        "\n",
        "> 🔑 **Note:** A matrix or tensor is typically represented by a capital letter (eg. `X` or `A`) where as a vector is typically represent by a lowercase letter (eg. `y` or `b`)"
      ],
      "metadata": {
        "id": "_KgMe1VYXoRf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "numpy_A = np.arange(1,29,dtype=np.int32) # create a NumPy array betweem 1 and 29\n",
        "numpy_A, numpy_A.size"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "w_O9u1q3Yec4",
        "outputId": "6b5762ce-261a-493d-84cc-c931f475dbff"
      },
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,\n",
              "        18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], dtype=int32), 28)"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tf.constant(numpy_A, shape=(2,2,7)) \n",
        "\n",
        "# note : the shape total (2,2,7) has to match the number of elements in the array i.e 28"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lwtGYoiHaWt6",
        "outputId": "deb67d76-9b05-4c49-afe1-022ef2b7ddd6"
      },
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 7), dtype=int32, numpy=\n",
              "array([[[ 1,  2,  3,  4,  5,  6,  7],\n",
              "        [ 8,  9, 10, 11, 12, 13, 14]],\n",
              "\n",
              "       [[15, 16, 17, 18, 19, 20, 21],\n",
              "        [22, 23, 24, 25, 26, 27, 28]]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Getting information from tensors\n",
        "\n",
        "* Shape: The length (number of elements) of each of the dimensions of a tensor\n",
        "* Size: The total number of items in the tensor\n",
        "* Rank: The number of tensor dimensions. A scalar has rank 0, a vector has rank 1, a matrix is rank 2, a tensor has rank n.\n",
        "* Axis or Dimension: A particular dimension of a tensor."
      ],
      "metadata": {
        "id": "Rd9YsC8_ad0i"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a rank 4 tensor (4 dimension)\n",
        "rank_4_tensor = tf.zeros([2,3,4,5])\n",
        "rank_4_tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3JXGpxOXbRLI",
        "outputId": "ce9744b7-1da6-4a91-d897-7da1010d6998"
      },
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3, 4, 5), dtype=float32, numpy=\n",
              "array([[[[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]]],\n",
              "\n",
              "\n",
              "       [[[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "rank_4_tensor.shape, tf.size(rank_4_tensor), rank_4_tensor.ndim,  tf.rank(rank_4_tensor)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lG2WxgLBdboY",
        "outputId": "defeb22d-aa53-43e8-ffbf-71fe04507f55"
      },
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(TensorShape([2, 3, 4, 5]),\n",
              " <tf.Tensor: shape=(), dtype=int32, numpy=120>,\n",
              " 4,\n",
              " <tf.Tensor: shape=(), dtype=int32, numpy=4>)"
            ]
          },
          "metadata": {},
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Get various attributes of tensor\n",
        "print(\"Datatype of every element:\", rank_4_tensor.dtype.name)\n",
        "print(\"Number of dimensions(rank):\", rank_4_tensor.ndim)\n",
        "print(\"Shape of tensor:\",rank_4_tensor.shape)\n",
        "print(\"Elements along axis 0 of tensor:\", rank_4_tensor.shape[0])\n",
        "print(\"Elements along last axis of tensor:\", rank_4_tensor.shape[1])\n",
        "print(\"Total number of elements in tensor:\",tf.size(rank_4_tensor).numpy())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Pjx4DTrNdl7_",
        "outputId": "a1adbf56-264a-451f-8536-2abd414c113a"
      },
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Datatype of every element: float32\n",
            "Number of dimensions(rank): 4\n",
            "Shape of tensor: (2, 3, 4, 5)\n",
            "Elements along axis 0 of tensor: 2\n",
            "Elements along last axis of tensor: 3\n",
            "Total number of elements in tensor: 120\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Indexing in Tensors\n",
        "\n",
        "We can index tensors just like Python Lists."
      ],
      "metadata": {
        "id": "OksLUyxsezIE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Get the first 2 items of each dimension\n",
        "rank_4_tensor[:2, :2, :2, :2]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lAiQLybhfnFa",
        "outputId": "5cfc2f0b-6034-44e0-9aeb-4f4c06c825c6"
      },
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 2, 2), dtype=float32, numpy=\n",
              "array([[[[0., 0.],\n",
              "         [0., 0.]],\n",
              "\n",
              "        [[0., 0.],\n",
              "         [0., 0.]]],\n",
              "\n",
              "\n",
              "       [[[0., 0.],\n",
              "         [0., 0.]],\n",
              "\n",
              "        [[0., 0.],\n",
              "         [0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# get the dimension from each index except for the final one\n",
        "rank_4_tensor[:1, :1, :1, :]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "I2nw6V6SfvXc",
        "outputId": "694b49c7-e3c3-4cd2-db66-45e8772e2403"
      },
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 1, 1, 5), dtype=float32, numpy=array([[[[0., 0., 0., 0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Adding dimensions to the existing tensor\n",
        "\n",
        "We can also add dimensions to our tensor whilst keeping the same information present using below methods,\n",
        "\n",
        "* `tf.newaxis`\n",
        "* `tf.expand_dims()`"
      ],
      "metadata": {
        "id": "LA4Qe8cAgCkD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a rank 2 tensor (2 dimensions)\n",
        "rank_2_tensor = tf.constant([\n",
        "                              \n",
        "                                [10,7],\n",
        "                                [3,4]\n",
        "                              \n",
        "                            ])\n",
        "rank_2_tensor.ndim, rank_2_tensor.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "F1vPLwrqgnWD",
        "outputId": "c5ae6bf7-c0fd-4407-ad55-8d07bf5db2bb"
      },
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(2, TensorShape([2, 2]))"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# now add an extra dimension (to the end)\n",
        "rank_3_tensor = rank_2_tensor[...,tf.newaxis] # in python \"...\" means \"all dimensions prior to\"\n",
        "rank_3_tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Vv4_vwqShAb7",
        "outputId": "5dd95365-3391-4366-db63-aa2cc7e9c571"
      },
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 1), dtype=int32, numpy=\n",
              "array([[[10],\n",
              "        [ 7]],\n",
              "\n",
              "       [[ 3],\n",
              "        [ 4]]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# using tf.expand_dims() at the beginnning\n",
        "tf.expand_dims(rank_2_tensor, axis = 0)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Yb0TMYdAhklL",
        "outputId": "796746b2-4805-4296-9fa3-841fa5a55367"
      },
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 2, 2), dtype=int32, numpy=\n",
              "array([[[10,  7],\n",
              "        [ 3,  4]]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Manipulating Tensors (tensor operations)\n",
        "\n",
        "Finding patterns in tensors (numerical representation of data) requires manipulating them.\n",
        "\n",
        "Again, when building models in TensorFlow, much of this pattern discovery is done for you.\n"
      ],
      "metadata": {
        "id": "0F3W4BQOiNfw"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 1. Basic Operations\n",
        "\n",
        "`+`, `-`, `*`, `/`"
      ],
      "metadata": {
        "id": "_MfXG5y-kJCm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# adding values to a tensor using the addition operator\n",
        "tensor = tf.constant([[2,4],\n",
        "                      [5,6]])\n",
        "tensor + 10"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yZN9zPtHi1WQ",
        "outputId": "79830379-d915-4251-f0b4-1f639c0b4e0b"
      },
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[12, 14],\n",
              "       [15, 16]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# since we used constant tensors, the original tensor is unchanges\n",
        "tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FV6hLk3JjAtf",
        "outputId": "62dde912-4149-4b72-b53c-140c9f0f8c24"
      },
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[2, 4],\n",
              "       [5, 6]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 36
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# other operators\n",
        "tensor - 10"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lZAn8EmfjB-g",
        "outputId": "88055dd1-d068-48f8-f9aa-a5eb640be5c5"
      },
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[-8, -6],\n",
              "       [-5, -4]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tensor * 10"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jXX2t3Arjcq3",
        "outputId": "aea3fe05-aa90-44a9-ffa1-fe6907931d6d"
      },
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[20, 40],\n",
              "       [50, 60]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tensor / 10"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Q5lnLrJSje09",
        "outputId": "438bf8a5-3976-495b-a065-f296538db9a9"
      },
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=float64, numpy=\n",
              "array([[0.2, 0.4],\n",
              "       [0.5, 0.6]])>"
            ]
          },
          "metadata": {},
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can also use the equivalent TensorFlow function. Using the TensorFlow function (where possible) has the advantage of being speed up later down the line when running as part of a [TensorFlow graph](https://www.tensorflow.org/tensorboard/graphs)"
      ],
      "metadata": {
        "id": "f3m2nMTRjgKO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "tf.multiply(tensor, 10)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Dx0Y-KnWj2GJ",
        "outputId": "44fe1241-d82c-4466-9d7c-e135d69bd52e"
      },
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[20, 40],\n",
              "       [50, 60]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 40
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 2. Matrix Multiplication"
      ],
      "metadata": {
        "id": "baaoXN_tj5Lf"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "One of the most common operations in machine learning algorithms is Matrix Multiplication\n",
        "\n",
        "TensorFlow implements this matric multiplication functionality in the `tf.matmul()` method.\n",
        "\n",
        "The main two rules for matrix multiplication to remember are:\n",
        "\n",
        "1. The inner dimensions must match\n",
        "- it means [3,`2`] @ [`3`,3], here the highlighted dimensions should match, in this example multiplication is not possible.\n",
        "- [3,`2`] @ [`2`,3], this will work\n",
        "\n",
        "2. The resulting matrix has the shape of the outer dimensions.\n",
        "- it means [`3`,2] @ [2,`3`], result would be [`3`,`3`]"
      ],
      "metadata": {
        "id": "U4oLB1YWkCaX"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "> 🔑 **Note** : `@` in Python, is the symbol of matrix multiplication."
      ],
      "metadata": {
        "id": "2DfU_J7ALsOv"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Matrix multiplication in TensorFlow\n",
        "tf.matmul(tensor, tensor)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Wt2bzRPTNYxq",
        "outputId": "4ea4a5ac-6fc0-4b32-ae7e-8c77c45c0bc2"
      },
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[24, 32],\n",
              "       [40, 56]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# matrix multiplication with Python operator @\n",
        "tensor @ tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tFXMLbfLOBbm",
        "outputId": "3bb15fc8-68ec-433f-fd35-34783a9d76c2"
      },
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[24, 32],\n",
              "       [40, 56]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 42
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Both of these worked because the shape of our Tensors were identical,\n",
        "\n",
        "Let's try if the shape is mis-matched"
      ],
      "metadata": {
        "id": "LDS7bSY-OICl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# create (3,2) tensor\n",
        "X = tf.constant([[1,2,3],\n",
        "                 [4,5,6]])\n",
        "\n",
        "# create another (3,2) tensor\n",
        "Y = tf.constant([[7,8,9],\n",
        "                 [10,11,12]])\n",
        "\n",
        "X,Y"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EL06cz0VOa59",
        "outputId": "ab87e094-b339-4214-9e51-68dd4c2afa18"
      },
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              " array([[1, 2, 3],\n",
              "        [4, 5, 6]], dtype=int32)>,\n",
              " <tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              " array([[ 7,  8,  9],\n",
              "        [10, 11, 12]], dtype=int32)>)"
            ]
          },
          "metadata": {},
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tf.matmul(X,Y)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 312
        },
        "id": "HantwLI4PK0d",
        "outputId": "f7afdc38-86c2-4005-dc5c-5bb09e52e808"
      },
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "error",
          "ename": "InvalidArgumentError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mInvalidArgumentError\u001b[0m                      Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-44-cc348e6d8216>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmatmul\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mY\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/traceback_utils.py\u001b[0m in \u001b[0;36merror_handler\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    151\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0mException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    152\u001b[0m       \u001b[0mfiltered_tb\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_process_traceback_frames\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__traceback__\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 153\u001b[0;31m       \u001b[0;32mraise\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwith_traceback\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfiltered_tb\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    154\u001b[0m     \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    155\u001b[0m       \u001b[0;32mdel\u001b[0m \u001b[0mfiltered_tb\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py\u001b[0m in \u001b[0;36mraise_from_not_ok_status\u001b[0;34m(e, name)\u001b[0m\n\u001b[1;32m   7162\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mraise_from_not_ok_status\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   7163\u001b[0m   \u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmessage\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;34m\" name: \"\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;34m\"\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 7164\u001b[0;31m   \u001b[0;32mraise\u001b[0m \u001b[0mcore\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_status_to_exception\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;32mNone\u001b[0m  \u001b[0;31m# pylint: disable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   7165\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   7166\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mInvalidArgumentError\u001b[0m: Matrix size-incompatible: In[0]: [2,3], In[1]: [2,3] [Op:MatMul]"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We're violating the first rule, which means the inner dimensions of two tensors must be same to be multiplied.\n",
        "\n",
        "So, now let's make it, and make the second tensor fit for multiplication using reshaping it."
      ],
      "metadata": {
        "id": "hVX-E1ENPNPm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# reshape the tensor\n",
        "print(tf.reshape(Y, shape=(3,2)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PlC-q-SLPdk3",
        "outputId": "0ebee31b-0145-4495-d40a-5f25b1ec2160"
      },
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tf.Tensor(\n",
            "[[ 7  8]\n",
            " [ 9 10]\n",
            " [11 12]], shape=(3, 2), dtype=int32)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# let's multiple now\n",
        "tf.matmul(X, tf.reshape(Y, shape=(3,2)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VUJTbXwgP_Pl",
        "outputId": "edbdc6f5-446d-43fc-bfb2-718ea229c64c"
      },
      "execution_count": 47,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 58,  64],\n",
              "       [139, 154]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 47
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We might have produced the result of multiplication, but notice `tf.reshape` has changed the order of the tensor, so instead we will use Transposing the tensor"
      ],
      "metadata": {
        "id": "XMNw6vCOQDZc"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# let's transpose and see\n",
        "tf.transpose(Y)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "m1A1zk6sQHLJ",
        "outputId": "ade2866a-7e6f-4870-a4e0-d1f8e0b165ae"
      },
      "execution_count": 48,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[ 7, 10],\n",
              "       [ 8, 11],\n",
              "       [ 9, 12]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# and then try the multiplication\n",
        "tf.matmul(X, tf.transpose(Y))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "khsD32B5Q4_F",
        "outputId": "d9e59d31-d219-4230-d6ab-88c046aac40a"
      },
      "execution_count": 49,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 50,  68],\n",
              "       [122, 167]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Voila ! See the difference in the result"
      ],
      "metadata": {
        "id": "zLhmmboTT4O_"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "There's one more way to perform matrix multiplication in TensorFlow is using `tf.tensordot()`"
      ],
      "metadata": {
        "id": "FER8jZoAUEWa"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# perform the dot product on X and Y\n",
        "tf.tensordot(X, tf.transpose(Y), axes = 1)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PvXPFkHWUNai",
        "outputId": "b34c3718-fe36-48eb-f836-cbd1ed6aea08"
      },
      "execution_count": 50,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 50,  68],\n",
              "       [122, 167]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 50
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Changing the datatype of the tensor\n",
        "\n",
        "Sometimes you'll want to alter the default datatype of your tensor.\n",
        "\n",
        "This is common when you want to compute using less precision (eg 16-bit floating point numbers vs 32-bit floating point numbers).\n",
        "\n",
        "Computing with less precision is useful on devices with less computing capacity such as mobile devices (because the less bit, the less space the computations require).\n",
        "\n",
        "We can change the datatype of a tensor using `tf.cast()`"
      ],
      "metadata": {
        "id": "nSefZAK6UWwS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a tensor with default datatype (float32)\n",
        "B = tf.constant([1.7,1.4])\n",
        "\n",
        "# Create a tensor with default datatype (int32)\n",
        "C = tf.constant([7,4])\n",
        "\n",
        "B,C"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fWcsf2Y2VOXo",
        "outputId": "5be7de6d-e2da-490f-b65c-218226395945"
      },
      "execution_count": 51,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2,), dtype=float32, numpy=array([1.7, 1.4], dtype=float32)>,\n",
              " <tf.Tensor: shape=(2,), dtype=int32, numpy=array([7, 4], dtype=int32)>)"
            ]
          },
          "metadata": {},
          "execution_count": 51
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# now let's cast and reduce the precision of both\n",
        "B = tf.cast(B, dtype=tf.float16)\n",
        "\n",
        "C = tf.cast(C, dtype=tf.int16)\n",
        "\n",
        "B,C"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_rq792TIWEkh",
        "outputId": "ecd631aa-d553-4aba-b883-20295456635b"
      },
      "execution_count": 52,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2,), dtype=float16, numpy=array([1.7, 1.4], dtype=float16)>,\n",
              " <tf.Tensor: shape=(2,), dtype=int16, numpy=array([7, 4], dtype=int16)>)"
            ]
          },
          "metadata": {},
          "execution_count": 52
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Getting the absolute value\n",
        "\n",
        "Sometimes you'll want the absolute values (all values are positive) of elements in your tensors\n",
        "\n",
        "\n",
        "To do so, you can use `tf.abs()`"
      ],
      "metadata": {
        "id": "YnplufOFWXeS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# create tensor with negative values\n",
        "D = tf.constant([-4, -7])\n",
        "D"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xzGeMSGNW11s",
        "outputId": "31c15933-ccd0-4d69-8b17-f556a9736e5f"
      },
      "execution_count": 53,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([-4, -7], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 53
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# get the absolute values\n",
        "tf.abs(D)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8QAqeIImW9A0",
        "outputId": "b06914f6-d575-4db3-feeb-1fb156cec20b"
      },
      "execution_count": 54,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([4, 7], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 54
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Aggregation Functions for Tensors (find min, max, mean and more)\n",
        "\n",
        "We can quickly aggregate(perform a calculation on a whole tensor) tensors to find things like the minimum value, maximum value, mean and sum of all the elements.\n",
        "\n",
        "\n",
        "To do so, aggregation methods typically have the syntax `reduce_[action]`, such as:\n",
        "\n",
        "* `tf.reduce_max()`\n",
        "* `tf.reduce_min()`\n",
        "* `tf.reduce_mean()`\n",
        "* `tf.reduce_sum()`\n",
        "* `tf.math.reduce_std()` <------------|Some functions needed to be use with \n",
        "* `tf.math.reduce_variance()` <---|`math` then followed by the function\n",
        "\n",
        "> 🔑 **Note**: Typically, each of these functions are under the `math` module, eg. `tf.math.reduce_min()` but we can use the alias `tf.reduce_min()`"
      ],
      "metadata": {
        "id": "BaB3_W7FXA4s"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create tensor with 50 random values between 0 and 100\n",
        "tf.random.set_seed(42)\n",
        "E = tf.random.uniform(shape = (50,),minval=0, maxval=1)\n",
        "# we can use this as well, tf.constant(np.random.randint(0,100, size = 50))\n",
        "E"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cyCkjdLrYOE8",
        "outputId": "d1f6434a-5e91-48ff-9da8-454706b9513b"
      },
      "execution_count": 55,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(50,), dtype=float32, numpy=\n",
              "array([0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
              "       0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
              "       0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
              "       0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
              "       0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
              "       0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
              "       0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
              "       0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
              "       0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
              "       0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043],\n",
              "      dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 55
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the minimum\n",
        "tf.reduce_min(E)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mhkR9FjPYhZh",
        "outputId": "76ff058e-4ea5-4e3c-eb1a-d29d708e322f"
      },
      "execution_count": 56,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.009463668>"
            ]
          },
          "metadata": {},
          "execution_count": 56
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the maximum\n",
        "tf.reduce_max(E)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "z99E3KbRZcKE",
        "outputId": "6515bbc0-6435-475e-837e-3e4ee79f00c5"
      },
      "execution_count": 57,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.9671384>"
            ]
          },
          "metadata": {},
          "execution_count": 57
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the mean\n",
        "tf.reduce_mean(E)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LdoGb9csZiw1",
        "outputId": "6ffa4456-f246-4cf3-8acb-21ff7c33d729"
      },
      "execution_count": 58,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.513808>"
            ]
          },
          "metadata": {},
          "execution_count": 58
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the sum\n",
        "tf.reduce_sum(E)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2K-x2-yLZlud",
        "outputId": "147e4874-de9d-4899-a23f-c6bf921cc2f4"
      },
      "execution_count": 59,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=25.690401>"
            ]
          },
          "metadata": {},
          "execution_count": 59
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the standard deviation\n",
        "tf.math.reduce_std(E)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "C2dlTe4-ZqcG",
        "outputId": "98d92061-a35d-4b3c-a338-99b306a8365e"
      },
      "execution_count": 60,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.2664255>"
            ]
          },
          "metadata": {},
          "execution_count": 60
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the variance\n",
        "tf.math.reduce_variance(E)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9VQN1L7ZZwNe",
        "outputId": "c917e30f-1049-4ac0-c997-070f8980b982"
      },
      "execution_count": 61,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.070982546>"
            ]
          },
          "metadata": {},
          "execution_count": 61
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Find the positional maximum and minimum\n",
        "\n",
        "How about finding the position a tensor where the maximum value occurs ?\n",
        "\n",
        "This is helpful when you want to line up your lables (say `['Green','Blue','Red']`) with your prediction probabilites tensor (eg `[0.98, 0.01, 0.01]`)\n",
        "\n",
        "In this case, the predicted label (the one with the highest prediction probability) would be `Green`\n",
        "\n",
        "You can do the same for the minimum(if required) with the following:\n",
        "\n",
        "* `tf.argmax()` - find the position of the maximum element in a given tensor\n",
        "* `tf.argmin()` - find the position of the minimum element in a given tensor"
      ],
      "metadata": {
        "id": "0nvFRBGkagnw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# so taking the same tensor from above\n",
        "E"
      ],
      "metadata": {
        "id": "reRAwUXNgZpX",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3bd54f20-8522-407f-caf6-45f6f7bc737d"
      },
      "execution_count": 63,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(50,), dtype=float32, numpy=\n",
              "array([0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
              "       0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
              "       0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
              "       0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
              "       0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
              "       0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
              "       0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
              "       0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
              "       0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
              "       0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043],\n",
              "      dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 63
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the maximum element in a given tensor\n",
        "tf.argmax(E)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MG5eatNkeM2j",
        "outputId": "450c0daf-5441-4b2a-a7b3-88f58b921cda"
      },
      "execution_count": 66,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=42>"
            ]
          },
          "metadata": {},
          "execution_count": 66
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# extended to find the position\n",
        "tf.argmax(E).numpy()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SGO499zLeUVW",
        "outputId": "df2d4cf5-ed1e-4f35-ada6-835fa537cbb7"
      },
      "execution_count": 67,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "42"
            ]
          },
          "metadata": {},
          "execution_count": 67
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# let's whether it's correct or not\n",
        "(tf.reduce_max(E).numpy() ==  E[tf.argmax(E).numpy()]).numpy()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dthIFlnjed3K",
        "outputId": "87a2c832-5163-4080-9eb2-184e9f3f40bd"
      },
      "execution_count": 73,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 73
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the minimum element in a given tensor\n",
        "tf.argmin(E), tf.reduce_min(E), E[tf.argmin(E)]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fYaPSqzge11b",
        "outputId": "6e48c432-2ea8-4cf6-8ca6-1e1cc40f0c08"
      },
      "execution_count": 75,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(), dtype=int64, numpy=16>,\n",
              " <tf.Tensor: shape=(), dtype=float32, numpy=0.009463668>,\n",
              " <tf.Tensor: shape=(), dtype=float32, numpy=0.009463668>)"
            ]
          },
          "metadata": {},
          "execution_count": 75
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Squeezing a tensor (removing all single dimensions)\n",
        "\n",
        "If you need to remove single-dimensions from a tensor (dimensions with size 1), you can use `tf.squeeze()`"
      ],
      "metadata": {
        "id": "aVm4ujCcf-eU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a 5 dimension tensor of 50 numbers between 0 and 100\n",
        "\n",
        "F = tf.constant(np.random.randint(0,100,50), shape=(1,1,1,1,50))\n",
        "\n",
        "F.shape, F.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DXkhVH4dgZu-",
        "outputId": "c9a3973c-9e05-42e2-e321-5f788b9b80b4"
      },
      "execution_count": 76,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(TensorShape([1, 1, 1, 1, 50]), 5)"
            ]
          },
          "metadata": {},
          "execution_count": 76
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "F_squeezed = tf.squeeze(F)"
      ],
      "metadata": {
        "id": "tya9v3pbguwN"
      },
      "execution_count": 77,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "F_squeezed.shape, F_squeezed.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_fXxfR94g0Ed",
        "outputId": "09694c6e-53db-41e7-8c33-39af46bda8dd"
      },
      "execution_count": 79,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(TensorShape([50]), 1)"
            ]
          },
          "metadata": {},
          "execution_count": 79
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### One Hot Encoding\n",
        "\n",
        "If you have a tensor of indices and would like to one-hot encode it, you can use `tf.one_hot()` - Basically Numerical encoding\n",
        "\n",
        "Specify `depth` parameter (the level which you want to one-hot encode to)."
      ],
      "metadata": {
        "id": "EFiIc3RPg2gl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a list of indices\n",
        "some_list = [0, 1, 2, 3]\n",
        "\n",
        "# One hot encode them\n",
        "tf.one_hot(some_list, depth = 4)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oxUJ5GtahYsD",
        "outputId": "6d7dab78-00e7-408c-85f2-68f87ffa562e"
      },
      "execution_count": 81,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4, 4), dtype=float32, numpy=\n",
              "array([[1., 0., 0., 0.],\n",
              "       [0., 1., 0., 0.],\n",
              "       [0., 0., 1., 0.],\n",
              "       [0., 0., 0., 1.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 81
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# We can also specify instead of 0 and 1, using on_value and off_value\n",
        "tf.one_hot(some_list, depth = 5, on_value=\"Yes\", off_value=\"No\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qAGYcnyiiR_3",
        "outputId": "7a51807d-9f79-45d4-c505-db2f3322f8b4"
      },
      "execution_count": 84,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4, 5), dtype=string, numpy=\n",
              "array([[b'Yes', b'No', b'No', b'No', b'No'],\n",
              "       [b'No', b'Yes', b'No', b'No', b'No'],\n",
              "       [b'No', b'No', b'Yes', b'No', b'No'],\n",
              "       [b'No', b'No', b'No', b'Yes', b'No']], dtype=object)>"
            ]
          },
          "metadata": {},
          "execution_count": 84
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Some more math functions\n",
        "\n",
        "* `tf.square()`\n",
        "* `tf.math.log()`\n",
        "* `tf.sqrt()`"
      ],
      "metadata": {
        "id": "hmWJzprRimcp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a tensor\n",
        "G = tf.constant(np.arange(1,10))\n",
        "G"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mi9ymbZGms4d",
        "outputId": "8bd5b35f-2656-4b8d-bc4c-b135c0546ae5"
      },
      "execution_count": 85,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=int64, numpy=array([1, 2, 3, 4, 5, 6, 7, 8, 9])>"
            ]
          },
          "metadata": {},
          "execution_count": 85
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Squaring\n",
        "tf.square(G)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RlqNukKCm2jN",
        "outputId": "dda01851-a30c-4128-ec32-11cecbffe777"
      },
      "execution_count": 86,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=int64, numpy=array([ 1,  4,  9, 16, 25, 36, 49, 64, 81])>"
            ]
          },
          "metadata": {},
          "execution_count": 86
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Log - it takes only non-integer\n",
        "tf.math.log(tf.cast(G, tf.float16))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NtiUZb8zm61N",
        "outputId": "1ebd905d-d951-4b76-89b3-2a33f942fb7f"
      },
      "execution_count": 89,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=float16, numpy=\n",
              "array([0.    , 0.6934, 1.099 , 1.387 , 1.609 , 1.792 , 1.946 , 2.08  ,\n",
              "       2.197 ], dtype=float16)>"
            ]
          },
          "metadata": {},
          "execution_count": 89
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Square root - it takes only non-integer\n",
        "tf.sqrt(tf.cast(G, tf.float16))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JH0MLTGDm-y9",
        "outputId": "f9e04002-952c-43ec-ab69-8b90eebd1c0b"
      },
      "execution_count": 91,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=float16, numpy=\n",
              "array([1.   , 1.414, 1.732, 2.   , 2.236, 2.45 , 2.646, 2.828, 3.   ],\n",
              "      dtype=float16)>"
            ]
          },
          "metadata": {},
          "execution_count": 91
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Tensors and Numpy\n",
        "\n",
        "We've seen some examples of tensors interact with Numpy arrays, such as, using NumPy arrays to create tensors.\n",
        "\n",
        "Tensors can also be converted to NumPy arrays using,\n",
        "\n",
        "* np.array()\n",
        "* tensor.numpy()"
      ],
      "metadata": {
        "id": "WtWUWf6EnMDt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a tensor using Numpy array\n",
        "H = tf.constant(np.array([3.,6.,9.]))\n",
        "H"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PsZ0Z_UZnro4",
        "outputId": "13c815c0-120c-47d9-f372-4bc0539335c2"
      },
      "execution_count": 93,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3,), dtype=float64, numpy=array([3., 6., 9.])>"
            ]
          },
          "metadata": {},
          "execution_count": 93
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Convert the H tensor into numpy using np.array()\n",
        "np.array(H), type(np.array(H))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YZbHCGLun1bG",
        "outputId": "7dfcf639-966d-4123-bf33-40f8496adedc"
      },
      "execution_count": 95,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([3., 6., 9.]), numpy.ndarray)"
            ]
          },
          "metadata": {},
          "execution_count": 95
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Convert the H tensor into numpy using tensor.numpy()\n",
        "H.numpy(), type(H.numpy())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VO1ur7bMn_Cn",
        "outputId": "b9af1afc-c046-4de1-bbb8-2918ceaa49ce"
      },
      "execution_count": 96,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([3., 6., 9.]), numpy.ndarray)"
            ]
          },
          "metadata": {},
          "execution_count": 96
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "By Default tensors have `float32` dtype whereas Numpy arrays have `float64`\n",
        "\n",
        "This is because neural networks can generally work very well with less precision (32-bit rather than 64-bit)"
      ],
      "metadata": {
        "id": "iYJ8qM-LoT0v"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a tensor from Numpy and from an array\n",
        "numpy_J = tf.constant(np.array([1.,2.,3.]))\n",
        "tensor_J = tf.constant([1.,2.,3.])\n",
        "\n",
        "numpy_J.dtype, tensor_J.dtype"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UqG62howo04x",
        "outputId": "6b7c589b-f6f1-4dcd-c83e-a20b4f806e2a"
      },
      "execution_count": 97,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tf.float64, tf.float32)"
            ]
          },
          "metadata": {},
          "execution_count": 97
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "tZQzU_dHpHvh"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}