02_numpy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Numpy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Python lists:\n",
    "\n",
    "* are very flexible\n",
    "* don't require uniform numerical types\n",
    "* are very easy to modify (inserting or appending objects).\n",
    "\n",
    "However, flexibility often comes at the cost of performance, and lists are not the ideal object for numerical calculations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "This is where **Numpy** comes in. Numpy is a Python module that defines a powerful n-dimensional array object that uses C and Fortran code behind the scenes to provide high performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.6883940696716309 seconds\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "import math\n",
    "\n",
    "a = range(10000000)\n",
    "\n",
    "def func(a):\n",
    "    return 1e-6*(4*a**2.) #7+1./(a+3)**3.4-(23*a)**4.2)-20\n",
    "\n",
    "# measure how long it takes in seconds\n",
    "start_time = time.time()\n",
    "\n",
    "new_a = []\n",
    "for val in a:\n",
    "    new_a.append(func(val))\n",
    "    \n",
    "print(f'{time.time()-start_time} seconds')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "range(0, 10000000)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.5598809719085693 seconds\n"
     ]
    }
   ],
   "source": [
    "import numpy\n",
    "\n",
    "start_time = time.time()\n",
    "a = numpy.array(a)\n",
    "new_a = func(a)\n",
    "print(f'{time.time()-start_time} seconds')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "209 µs ± 3.61 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)\n",
      "700 ns ± 7.5 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)\n"
     ]
    }
   ],
   "source": [
    "L = range(1000)\n",
    "%timeit [i**2 for i in L]\n",
    "\n",
    "a = np.arange(1000)\n",
    "%timeit a**2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "The downside of Numpy arrays is that they have a more rigid structure, and require a single numerical type (e.g. floating point values), but for a lot of scientific work, this is exactly what is needed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "The Numpy module is imported with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Although in the rest of this course, and in many packages, the following convention is used:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is because Numpy is so often used that it is shorter to type ``np`` than ``numpy``."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating Numpy arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "The easiest way to create an array is from a Python list, using the ``array`` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = np.array([10, 20.1, 30, 40])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 2, 3, 4])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b = list()\n",
    "for n in range(5):\n",
    "    b.append(n)\n",
    "c = np.array(b)\n",
    "c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "numpy.ndarray"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Numpy arrays have several attributes that give useful information about the array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a.ndim  # number of dimensions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4,)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a.shape  # shape of the array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('float64')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a.dtype  # numerical type"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "There are several other ways to create arrays. For example, there is an ``arange`` function that can be used similarly to the built-in Python ``range`` function, with the exception that it can take floating-point input:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 2, 3, 4, 5])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d =np.arange(1, 6, 1)\n",
    "d\n",
    "# np.arange(0, 10, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('int64')"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.arange(30.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "np.arange(3, 12, 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "b = np.arange(1.2, 4.4, 0.1)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2. , 2.1, 2.2, 2.3, 2.4,\n",
       "       2.5, 2.6, 2.7, 2.8, 2.9, 3. , 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7,\n",
       "       3.8, 3.9, 4. , 4.1, 4.2, 4.3])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.5000000000000002"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b[3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Another useful function is ``linspace``, which can be used to create linearly spaced values between and including limits:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([11.        , 11.11111111, 11.22222222, 11.33333333, 11.44444444,\n",
       "       11.55555556, 11.66666667, 11.77777778, 11.88888889, 12.        ])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.linspace(11, 12, 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "and a similar function can be used to create logarithmically spaced values between and including limits:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([   10.        ,    31.6227766 ,   100.        ,   316.22776602,\n",
       "        1000.        ,  3162.27766017, 10000.        ])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.logspace(1., 4., 7)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Finally, the ``zeros`` and ``ones`` functions can be used to create arrays intially set to ``0`` and ``1`` respectively:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.zeros(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 1.],\n",
       "       [1., 1.],\n",
       "       [1., 1.]])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.ones([5, 3, 5])[0, :, 1:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "np.array([X, X, X, X])\n",
    "\n",
    "np.arange(start, finish, step)\n",
    "\n",
    "np.linspace(start, finish-included, number of elements)\n",
    "\n",
    "np.zeros([dim, dim]) 1D: np.zeros(X)\n",
    "\n",
    "np.ones([dim, dim])\n",
    "\n",
    "np.empty([dim, dim])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([10. , 20.1, 30. , 40. ])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2. , 2.1, 2.2, 2.3, 2.4,\n",
       "       2.5, 2.6, 2.7, 2.8, 2.9, 3. , 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7,\n",
       "       3.8, 3.9, 4. , 4.1, 4.2, 4.3, 4.4])"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2, 4)"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c = np.vstack([a, a]) ### stack, vstack, hstack\n",
    "c.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2, 4)"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.shape(c)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[10. , 20.1, 30. , 40. ],\n",
       "       [10. , 20.1, 30. , 40. ]])"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d = np.vstack([a, a])\n",
    "d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "e = np.hstack([a, b])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(37,)"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.shape(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create an array which contains the value 2 repeated 10 times"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2., 2., 2., 2., 2., 2., 2., 2., 2., 2.])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.ones(10) + 1 # * 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2, 2, 2, 2, 2, 2, 2, 2, 2, 2])"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.array([2 for i in range(10)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2., 2., 2., 2., 2., 2., 2., 2., 2., 2.])"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.zeros(10) + 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create an array which contains values from 1 until 90 every one and then the values 95, 99, 99.9, 99.99."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1.  ,  2.  ,  3.  ,  4.  ,  5.  ,  6.  ,  7.  ,  8.  ,  9.  ,\n",
       "       10.  , 11.  , 12.  , 13.  , 14.  , 15.  , 16.  , 17.  , 18.  ,\n",
       "       19.  , 20.  , 21.  , 22.  , 23.  , 24.  , 25.  , 26.  , 27.  ,\n",
       "       28.  , 29.  , 30.  , 31.  , 32.  , 33.  , 34.  , 35.  , 36.  ,\n",
       "       37.  , 38.  , 39.  , 40.  , 41.  , 42.  , 43.  , 44.  , 45.  ,\n",
       "       46.  , 47.  , 48.  , 49.  , 50.  , 51.  , 52.  , 53.  , 54.  ,\n",
       "       55.  , 56.  , 57.  , 58.  , 59.  , 60.  , 61.  , 62.  , 63.  ,\n",
       "       64.  , 65.  , 66.  , 67.  , 68.  , 69.  , 70.  , 71.  , 72.  ,\n",
       "       73.  , 74.  , 75.  , 76.  , 77.  , 78.  , 79.  , 80.  , 81.  ,\n",
       "       82.  , 83.  , 84.  , 85.  , 86.  , 87.  , 88.  , 89.  , 90.  ,\n",
       "       95.  , 99.  , 99.9 , 99.99])"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "v1 = np.arange(1, 90.1, 1)  \n",
    "v2 = [95, 99, 99.9, 99.99]\n",
    "np.hstack([v1, v2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,\n",
       "       18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34,\n",
       "       35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51,\n",
       "       52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68,\n",
       "       69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85,\n",
       "       86, 87, 88, 89, 90])"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.array([i for i in range(1, 91)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Numerical operations with arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Numpy arrays can be combined numerically using the standard ``+-*/**`` operators:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "x1 = np.array([1,2,3])\n",
    "y1 = np.array([4,5,6])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([4, 5, 6])"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 8, 10, 12])"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "2 * y1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([12, 20, 30])"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(x1 + 2) * y1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  1,  32, 729])"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x1 ** y1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Note that this differs from lists:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = [1,2,3]\n",
    "y = [4,5,6]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[4, 5, 6, 'hola']"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.append('hola')\n",
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[4, 5, 6, 'hola', 4, 5, 6, 'hola', 4, 5, 6, 'hola']"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "3 * y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[4, 5, 6, 'hola', 4, 5, 6, 'hola']"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "2 * y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 2, 3, 4, 5, 6, 'hola', 4, 5, 6, 'hola']"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x + 2 * y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Accessing and Slicing Arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Similarly to lists, items in arrays can be accessed individually:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.array([9,8,7])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([9, 8, 7])"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "and arrays can also be **sliced** by specifiying the start and end of the slice (where the last element is exclusive):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([10, 11, 12, 13, 14, 15, 16, 17, 18, 19])"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = np.arange(10, 20)\n",
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([10, 12, 14])"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y[0:5:2] # Slices [start:end:step] [:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "optionally specifying a step:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "y[0:10:2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "As for lists, the start, end, and step are all optional, and default to ``0``, ``len(array)``, and ``1`` respectively:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([10, 11, 12, 13, 14])"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([10, 12, 14, 16, 18])"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y[::2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Given an array ``x`` with 20 elements, find the array ``dx`` containing 19 values where ``dx[i] = x[i+1] - x[i]``. Do this without loops!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "x = [3, 6, 7, 8]\n",
    "\n",
    "l = list()\n",
    "\n",
    "for ii, xi in enumerate(x[:-1]):\n",
    "    l.append(x[ii+1] - xi)\n",
    "\n",
    "\n",
    "for ii in range(len(x)-1):\n",
    "    l.append(x[ii+1] - x[ii])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([88, 88, 81, 79, 56])"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = np.random.randint(10, 100, 5)\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  0,  -7,  -2, -23])"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[1:, 1:] - x[:-1, :-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  0,  -7,  -2, -23])"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.diff(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multi-dimensional arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center> <img src=\"img/numpy_indexing.png\" width=\"1600\"/> </center>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "y = np.ones([3,2,3])  # ones takes the shape of the array, not the values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3, 2, 3)"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[1., 1., 1.],\n",
       "        [1., 1., 1.]],\n",
       "\n",
       "       [[1., 1., 1.],\n",
       "        [1., 1., 1.]],\n",
       "\n",
       "       [[1., 1., 1.],\n",
       "        [1., 1., 1.]]])"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Multi-dimensional arrays can be sliced differently along different dimensions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [],
   "source": [
    "z = np.ones([6,6,6])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[1., 1., 1., 1., 1., 1.],\n",
       "        [1., 1., 1., 1., 1., 1.],\n",
       "        [1., 1., 1., 1., 1., 1.]],\n",
       "\n",
       "       [[1., 1., 1., 1., 1., 1.],\n",
       "        [1., 1., 1., 1., 1., 1.],\n",
       "        [1., 1., 1., 1., 1., 1.]]])"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z[::3, 1:4, :]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Constants"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "NumPy provides us access to some useful constants as well - remember you should never be typing these in manually! Other libraries such as SciPy and MetPy have their own set of constants that are more domain specific."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.141592653589793"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.pi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.718281828459045"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.e"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4.141592653589793"
      ]
     },
     "execution_count": 112,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "1 + np.pi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "In addition to an array class, Numpy contains a number of **vectorized** functions, which means functions that can act on all the elements of an array, typically much faster than could be achieved by looping over the array."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "theta = np.linspace(0., 2. * np.pi, 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.        , 0.6981317 , 1.3962634 , 2.0943951 , 2.7925268 ,\n",
       "       3.4906585 , 4.1887902 , 4.88692191, 5.58505361, 6.28318531])"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "theta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10,)"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.shape(theta)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.283185307179586"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "theta.max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.283185307179586"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.max(theta)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2, 10)"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tt = np.vstack([theta, theta])\n",
    "tt.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.283185307179586"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tt.max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "aa = np.array([np.linspace(0., 2. * np.pi, 10), np.linspace(0., 2. * np.pi, 10)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([0.        , 0.6981317 , 1.3962634 , 2.0943951 , 2.7925268 ,\n",
       "        3.4906585 , 4.1887902 , 4.88692191, 5.58505361, 6.28318531]),\n",
       " array([0.        , 0.6981317 , 1.3962634 , 2.0943951 , 2.7925268 ,\n",
       "        3.4906585 , 4.1887902 , 4.88692191, 5.58505361, 6.28318531])]"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pp = [np.linspace(0., 2. * np.pi, 10), np.linspace(0., 2. * np.pi, 10)]\n",
    "pp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(pp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "aa = np.asarray(pp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "aa.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.00000000e+00,  6.42787610e-01,  9.84807753e-01,\n",
       "         8.66025404e-01,  3.42020143e-01, -3.42020143e-01,\n",
       "        -8.66025404e-01, -9.84807753e-01, -6.42787610e-01,\n",
       "        -2.44929360e-16],\n",
       "       [ 0.00000000e+00,  6.42787610e-01,  9.84807753e-01,\n",
       "         8.66025404e-01,  3.42020143e-01, -3.42020143e-01,\n",
       "        -8.66025404e-01, -9.84807753e-01, -6.42787610e-01,\n",
       "        -2.44929360e-16]])"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sin(tt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sin(aa[0, 0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Another useful package is the ``np.random`` sub-package, which can be used to genenerate random numbers fast:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.48849716, 0.52766269, 0.83970123, 0.85602101, 0.64405742,\n",
       "       0.54420396, 0.65176724, 0.44591857, 0.1544168 , 0.2488368 ])"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# uniform distribution between 0 and 1\n",
    "np.random.random(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2.61840403, 2.16555013, 4.04433245, 5.64950756, 3.73826699,\n",
       "       2.84408076, 2.61227934, 2.99571071, 2.28552334, 3.71846935])"
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 10 values from a gaussian distribution with mean 3 and sigma 1\n",
    "np.random.normal(3., 1., 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0,  1,  2,  3],\n",
       "       [ 4,  5,  6,  7],\n",
       "       [ 8,  9, 10, 11]])"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = np.arange(12).reshape(3, 4)\n",
    "a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0,  1,  2,  3],\n",
       "       [ 4,  5,  6,  7],\n",
       "       [ 8,  9, 10, 11]])"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([12, 15, 18, 21])"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sum(a, axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.sum(a, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Another very useful function in Numpy is [numpy.loadtxt](http://docs.scipy.org/doc/numpy/reference/generated/numpy.loadtxt.html) which makes it easy to read in data from column-based data. For example, given the following file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "dir_data = Path('data')\n",
    "data = np.loadtxt(dir_data / 'columns.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[0;31mSignature:\u001b[0m\n",
       "\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mloadtxt\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mfname\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mdtype\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m<\u001b[0m\u001b[0;32mclass\u001b[0m \u001b[0;34m'float'\u001b[0m\u001b[0;34m>\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mcomments\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'#'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mdelimiter\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mconverters\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mskiprows\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0musecols\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0munpack\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mndmin\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mencoding\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'bytes'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mmax_rows\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;34m*\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mquotechar\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mlike\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
       "\u001b[0;31mDocstring:\u001b[0m\n",
       "Load data from a text file.\n",
       "\n",
       "Parameters\n",
       "----------\n",
       "fname : file, str, pathlib.Path, list of str, generator\n",
       "    File, filename, list, or generator to read.  If the filename\n",
       "    extension is ``.gz`` or ``.bz2``, the file is first decompressed. Note\n",
       "    that generators must return bytes or strings. The strings\n",
       "    in a list or produced by a generator are treated as lines.\n",
       "dtype : data-type, optional\n",
       "    Data-type of the resulting array; default: float.  If this is a\n",
       "    structured data-type, the resulting array will be 1-dimensional, and\n",
       "    each row will be interpreted as an element of the array.  In this\n",
       "    case, the number of columns used must match the number of fields in\n",
       "    the data-type.\n",
       "comments : str or sequence of str or None, optional\n",
       "    The characters or list of characters used to indicate the start of a\n",
       "    comment. None implies no comments. For backwards compatibility, byte\n",
       "    strings will be decoded as 'latin1'. The default is '#'.\n",
       "delimiter : str, optional\n",
       "    The character used to separate the values. For backwards compatibility,\n",
       "    byte strings will be decoded as 'latin1'. The default is whitespace.\n",
       "\n",
       "    .. versionchanged:: 1.23.0\n",
       "       Only single character delimiters are supported. Newline characters\n",
       "       cannot be used as the delimiter.\n",
       "\n",
       "converters : dict or callable, optional\n",
       "    Converter functions to customize value parsing. If `converters` is\n",
       "    callable, the function is applied to all columns, else it must be a\n",
       "    dict that maps column number to a parser function.\n",
       "    See examples for further details.\n",
       "    Default: None.\n",
       "\n",
       "    .. versionchanged:: 1.23.0\n",
       "       The ability to pass a single callable to be applied to all columns\n",
       "       was added.\n",
       "\n",
       "skiprows : int, optional\n",
       "    Skip the first `skiprows` lines, including comments; default: 0.\n",
       "usecols : int or sequence, optional\n",
       "    Which columns to read, with 0 being the first. For example,\n",
       "    ``usecols = (1,4,5)`` will extract the 2nd, 5th and 6th columns.\n",
       "    The default, None, results in all columns being read.\n",
       "\n",
       "    .. versionchanged:: 1.11.0\n",
       "        When a single column has to be read it is possible to use\n",
       "        an integer instead of a tuple. E.g ``usecols = 3`` reads the\n",
       "        fourth column the same way as ``usecols = (3,)`` would.\n",
       "unpack : bool, optional\n",
       "    If True, the returned array is transposed, so that arguments may be\n",
       "    unpacked using ``x, y, z = loadtxt(...)``.  When used with a\n",
       "    structured data-type, arrays are returned for each field.\n",
       "    Default is False.\n",
       "ndmin : int, optional\n",
       "    The returned array will have at least `ndmin` dimensions.\n",
       "    Otherwise mono-dimensional axes will be squeezed.\n",
       "    Legal values: 0 (default), 1 or 2.\n",
       "\n",
       "    .. versionadded:: 1.6.0\n",
       "encoding : str, optional\n",
       "    Encoding used to decode the inputfile. Does not apply to input streams.\n",
       "    The special value 'bytes' enables backward compatibility workarounds\n",
       "    that ensures you receive byte arrays as results if possible and passes\n",
       "    'latin1' encoded strings to converters. Override this value to receive\n",
       "    unicode arrays and pass strings as input to converters.  If set to None\n",
       "    the system default is used. The default value is 'bytes'.\n",
       "\n",
       "    .. versionadded:: 1.14.0\n",
       "max_rows : int, optional\n",
       "    Read `max_rows` rows of content after `skiprows` lines. The default is\n",
       "    to read all the rows. Note that empty rows containing no data such as\n",
       "    empty lines and comment lines are not counted towards `max_rows`,\n",
       "    while such lines are counted in `skiprows`.\n",
       "\n",
       "    .. versionadded:: 1.16.0\n",
       "\n",
       "    .. versionchanged:: 1.23.0\n",
       "        Lines containing no data, including comment lines (e.g., lines\n",
       "        starting with '#' or as specified via `comments`) are not counted\n",
       "        towards `max_rows`.\n",
       "quotechar : unicode character or None, optional\n",
       "    The character used to denote the start and end of a quoted item.\n",
       "    Occurrences of the delimiter or comment characters are ignored within\n",
       "    a quoted item. The default value is ``quotechar=None``, which means\n",
       "    quoting support is disabled.\n",
       "\n",
       "    If two consecutive instances of `quotechar` are found within a quoted\n",
       "    field, the first is treated as an escape character. See examples.\n",
       "\n",
       "    .. versionadded:: 1.23.0\n",
       "like : array_like, optional\n",
       "    Reference object to allow the creation of arrays which are not\n",
       "    NumPy arrays. If an array-like passed in as ``like`` supports\n",
       "    the ``__array_function__`` protocol, the result will be defined\n",
       "    by it. In this case, it ensures the creation of an array object\n",
       "    compatible with that passed in via this argument.\n",
       "\n",
       "    .. versionadded:: 1.20.0\n",
       "\n",
       "Returns\n",
       "-------\n",
       "out : ndarray\n",
       "    Data read from the text file.\n",
       "\n",
       "See Also\n",
       "--------\n",
       "load, fromstring, fromregex\n",
       "genfromtxt : Load data with missing values handled as specified.\n",
       "scipy.io.loadmat : reads MATLAB data files\n",
       "\n",
       "Notes\n",
       "-----\n",
       "This function aims to be a fast reader for simply formatted files.  The\n",
       "`genfromtxt` function provides more sophisticated handling of, e.g.,\n",
       "lines with missing values.\n",
       "\n",
       "Each row in the input text file must have the same number of values to be\n",
       "able to read all values. If all rows do not have same number of values, a\n",
       "subset of up to n columns (where n is the least number of values present\n",
       "in all rows) can be read by specifying the columns via `usecols`.\n",
       "\n",
       ".. versionadded:: 1.10.0\n",
       "\n",
       "The strings produced by the Python float.hex method can be used as\n",
       "input for floats.\n",
       "\n",
       "Examples\n",
       "--------\n",
       ">>> from io import StringIO   # StringIO behaves like a file object\n",
       ">>> c = StringIO(\"0 1\\n2 3\")\n",
       ">>> np.loadtxt(c)\n",
       "array([[0., 1.],\n",
       "       [2., 3.]])\n",
       "\n",
       ">>> d = StringIO(\"M 21 72\\nF 35 58\")\n",
       ">>> np.loadtxt(d, dtype={'names': ('gender', 'age', 'weight'),\n",
       "...                      'formats': ('S1', 'i4', 'f4')})\n",
       "array([(b'M', 21, 72.), (b'F', 35, 58.)],\n",
       "      dtype=[('gender', 'S1'), ('age', '<i4'), ('weight', '<f4')])\n",
       "\n",
       ">>> c = StringIO(\"1,0,2\\n3,0,4\")\n",
       ">>> x, y = np.loadtxt(c, delimiter=',', usecols=(0, 2), unpack=True)\n",
       ">>> x\n",
       "array([1., 3.])\n",
       ">>> y\n",
       "array([2., 4.])\n",
       "\n",
       "The `converters` argument is used to specify functions to preprocess the\n",
       "text prior to parsing. `converters` can be a dictionary that maps\n",
       "preprocessing functions to each column:\n",
       "\n",
       ">>> s = StringIO(\"1.618, 2.296\\n3.141, 4.669\\n\")\n",
       ">>> conv = {\n",
       "...     0: lambda x: np.floor(float(x)),  # conversion fn for column 0\n",
       "...     1: lambda x: np.ceil(float(x)),  # conversion fn for column 1\n",
       "... }\n",
       ">>> np.loadtxt(s, delimiter=\",\", converters=conv)\n",
       "array([[1., 3.],\n",
       "       [3., 5.]])\n",
       "\n",
       "`converters` can be a callable instead of a dictionary, in which case it\n",
       "is applied to all columns:\n",
       "\n",
       ">>> s = StringIO(\"0xDE 0xAD\\n0xC0 0xDE\")\n",
       ">>> import functools\n",
       ">>> conv = functools.partial(int, base=16)\n",
       ">>> np.loadtxt(s, converters=conv)\n",
       "array([[222., 173.],\n",
       "       [192., 222.]])\n",
       "\n",
       "This example shows how `converters` can be used to convert a field\n",
       "with a trailing minus sign into a negative number.\n",
       "\n",
       ">>> s = StringIO('10.01 31.25-\\n19.22 64.31\\n17.57- 63.94')\n",
       ">>> def conv(fld):\n",
       "...     return -float(fld[:-1]) if fld.endswith(b'-') else float(fld)\n",
       "...\n",
       ">>> np.loadtxt(s, converters=conv)\n",
       "array([[ 10.01, -31.25],\n",
       "       [ 19.22,  64.31],\n",
       "       [-17.57,  63.94]])\n",
       "\n",
       "Using a callable as the converter can be particularly useful for handling\n",
       "values with different formatting, e.g. floats with underscores:\n",
       "\n",
       ">>> s = StringIO(\"1 2.7 100_000\")\n",
       ">>> np.loadtxt(s, converters=float)\n",
       "array([1.e+00, 2.7e+00, 1.e+05])\n",
       "\n",
       "This idea can be extended to automatically handle values specified in\n",
       "many different formats:\n",
       "\n",
       ">>> def conv(val):\n",
       "...     try:\n",
       "...         return float(val)\n",
       "...     except ValueError:\n",
       "...         return float.fromhex(val)\n",
       ">>> s = StringIO(\"1, 2.5, 3_000, 0b4, 0x1.4000000000000p+2\")\n",
       ">>> np.loadtxt(s, delimiter=\",\", converters=conv, encoding=None)\n",
       "array([1.0e+00, 2.5e+00, 3.0e+03, 1.8e+02, 5.0e+00])\n",
       "\n",
       "Note that with the default ``encoding=\"bytes\"``, the inputs to the\n",
       "converter function are latin-1 encoded byte strings. To deactivate the\n",
       "implicit encoding prior to conversion, use ``encoding=None``\n",
       "\n",
       ">>> s = StringIO('10.01 31.25-\\n19.22 64.31\\n17.57- 63.94')\n",
       ">>> conv = lambda x: -float(x[:-1]) if x.endswith('-') else float(x)\n",
       ">>> np.loadtxt(s, converters=conv, encoding=None)\n",
       "array([[ 10.01, -31.25],\n",
       "       [ 19.22,  64.31],\n",
       "       [-17.57,  63.94]])\n",
       "\n",
       "Support for quoted fields is enabled with the `quotechar` parameter.\n",
       "Comment and delimiter characters are ignored when they appear within a\n",
       "quoted item delineated by `quotechar`:\n",
       "\n",
       ">>> s = StringIO('\"alpha, #42\", 10.0\\n\"beta, #64\", 2.0\\n')\n",
       ">>> dtype = np.dtype([(\"label\", \"U12\"), (\"value\", float)])\n",
       ">>> np.loadtxt(s, dtype=dtype, delimiter=\",\", quotechar='\"')\n",
       "array([('alpha, #42', 10.), ('beta, #64',  2.)],\n",
       "      dtype=[('label', '<U12'), ('value', '<f8')])\n",
       "\n",
       "Quoted fields can be separated by multiple whitespace characters:\n",
       "\n",
       ">>> s = StringIO('\"alpha, #42\"       10.0\\n\"beta, #64\" 2.0\\n')\n",
       ">>> dtype = np.dtype([(\"label\", \"U12\"), (\"value\", float)])\n",
       ">>> np.loadtxt(s, dtype=dtype, delimiter=None, quotechar='\"')\n",
       "array([('alpha, #42', 10.), ('beta, #64',  2.)],\n",
       "      dtype=[('label', '<U12'), ('value', '<f8')])\n",
       "\n",
       "Two consecutive quote characters within a quoted field are treated as a\n",
       "single escaped character:\n",
       "\n",
       ">>> s = StringIO('\"Hello, my name is \"\"Monty\"\"!\"')\n",
       ">>> np.loadtxt(s, dtype=\"U\", delimiter=\",\", quotechar='\"')\n",
       "array('Hello, my name is \"Monty\"!', dtype='<U26')\n",
       "\n",
       "Read subset of columns when all rows do not contain equal number of values:\n",
       "\n",
       ">>> d = StringIO(\"1 2\\n2 4\\n3 9 12\\n4 16 20\")\n",
       ">>> np.loadtxt(d, usecols=(0, 1))\n",
       "array([[ 1.,  2.],\n",
       "       [ 2.,  4.],\n",
       "       [ 3.,  9.],\n",
       "       [ 4., 16.]])\n",
       "\u001b[0;31mFile:\u001b[0m      /opt/miniforge3/envs/pangeo/lib/python3.10/site-packages/numpy/lib/npyio.py\n",
       "\u001b[0;31mType:\u001b[0m      function"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "np.loadtxt?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Masking"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "The index notation ``[...]`` is not limited to single element indexing, or multiple element slicing, but one can also pass a discrete list/array of indices:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.array([1,6,4,7, 9, 8, 10])\n",
    "#x[[True, False, True, False]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1,  6,  4,  7,  9,  8, 10])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "which is returning a new array composed of elements 1, 2, 4, etc from the original array."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Alternatively, one can also pass a boolean array of ``True/False`` values, called a **mask**, indicating which items to keep:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = np.array([3, 4, 5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([3, 4, 5])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "mask = np.array([True, False, False])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ True, False, False])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mask"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([3])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y[mask]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Now this doesn't look very useful because it is very verbose, but now consider that carrying out a comparison with the array will return such a boolean array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1,  6,  4,  7,  9,  8, 10])"
      ]
     },
     "execution_count": 141,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "mask = x > 3.4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 6,  4,  7,  9,  8, 10])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[mask]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "It is therefore possible to extract subsets from an array using the following simple notation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 6,  4,  7,  9,  8, 10])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mask = x> 3.4\n",
    "x[x> 3.4]\n",
    "# p[mask] "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 6,  4,  7,  9,  8, 10])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[mask]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1])"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[x <= 3.4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[~mask]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Conditions can be combined:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conditional formating\n",
    "\n",
    "##### Loops\n",
    "and, or\n",
    "\n",
    "#### Masking in numpy array\n",
    "& (and), | (or)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1,  6,  4,  7,  9,  8, 10])"
      ]
     },
     "execution_count": 152,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 6, 4, 7, 9, 8])"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[((x > 3.4) & (x < 9.5)) | (x<2)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Of course, the boolean **mask** can be derived from a different array to ``x`` as long as it is the right size:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.linspace(-1., 1., 14)\n",
    "y = np.array([1,6,4,7,9,3,1,5,6,7,3,4,4,3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(14,)"
      ]
     },
     "execution_count": 155,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(14,)"
      ]
     },
     "execution_count": 156,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([10, 11, 12])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y2 = y + 7\n",
    "y2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "mask = y2 > 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([False,  True,  True])"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mask"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([4, 5])"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y[mask]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 6, 4, 7, 9, 3, 1, 5, 6, 7, 3, 4, 4, 3])"
      ]
     },
     "execution_count": 160,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y[(x > -0.5) | (x < 0.4)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Since the mask itself is an array, it can be stored in a variable and used as a mask for different arrays:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "metadata": {},
   "outputs": [],
   "source": [
    "keep = (x > -0.5) & (x < 0.4)\n",
    "x_new = x[keep]\n",
    "y_new = y[keep]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "keep"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_new"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_new"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "we can use this conditional indexing to assign new values to certain positions within our array, somewhat like a masking operation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 6, 4, 7, 9, 3, 1, 5, 6, 7, 3, 4, 4, 3])"
      ]
     },
     "execution_count": 161,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([False,  True, False,  True,  True, False, False, False,  True,\n",
       "        True, False, False, False, False])"
      ]
     },
     "execution_count": 163,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mask = y>5\n",
    "mask"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "y[mask] = 999"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  3, 999, 999])"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.2, nan, nan, nan, nan, 3.2, 1.2, nan, nan, nan, 3.2, nan, nan,\n",
       "       3.2])"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = y + 0.1\n",
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "mm = y > 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([False,  True,  True,  True,  True, False, False,  True,  True,\n",
       "        True, False,  True,  True, False])"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.1, nan, nan, nan, nan, 3.1, 1.1, nan, nan, nan, 3.1, nan, nan,\n",
       "       3.1])"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y[mm] = np.NaN\n",
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([nan,  3.,  4., nan])"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pp = np.array([np.NaN, 3, 4, np.NaN])\n",
    "pp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y[y > 5] = 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### NaN values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In arrays, some of the values are sometimes NaN - meaning *Not a Number*. If you multiply a NaN value by another value, you get NaN, and if there are any NaN values in a summation, the total result will be NaN. One way to get around this is to use ``np.nansum`` instead of ``np.sum`` in order to find the sum:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1.,  2.,  3., nan])"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = np.array([1,2,3,np.NaN])\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "nan"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sum(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "nan"
      ]
     },
     "execution_count": 171,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.NAN # np.nan | np.NaN | np.NAN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.0"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.nansum(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.nansum(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.0"
      ]
     },
     "execution_count": 175,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.nanmax(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also use ``np.isnan`` to tell you where values are NaN. For example, ``array[~np.isnan(array)]`` will return all the values that are not NaN (because ~ means 'not'):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([False, False, False,  True])"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.isnan(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([nan])"
      ]
     },
     "execution_count": 177,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[np.isnan(x)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1., 2., 3.])"
      ]
     },
     "execution_count": 178,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[~np.isnan(x)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Statistics --> Scipy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy.random as rnd\n",
    "\n",
    "\n",
    "g = rnd.normal(loc=0, scale=1, size=1000000)\n",
    "\n",
    "print(numpy.mean(g), numpy.median(g), numpy.std(g))\n",
    "\n",
    "# specifying axis of operation gives different results:\n",
    "a = [[1,1,1], [2,2,2], [3,3,3]]\n",
    "print(numpy.mean(a))         # mean of all numbers in the array\n",
    "print(numpy.mean(a, axis=0)) # mean along axis 0 (first axis = outermost axis = along columns)\n",
    "print(numpy.mean(a, axis=1)) # mean along axis 1 (second axis = along rows)\n",
    "\n",
    "# operations that ignore nans\n",
    "b = [1, 2, 3, numpy.nan, 4, 5]\n",
    "print(numpy.mean(b))    # returns nan\n",
    "print(numpy.nanmean(b)) # ignores nan; see nanmedian, nanstd, ...\n",
    "\n",
    "# determine percentiles\n",
    "print(numpy.percentile(g, 50)) # the same as median\n",
    "print(numpy.percentile(g, 68.27)-numpy.percentile(g, 31.73))\n",
    "\n",
    "# create a histogram\n",
    "hist, bins = numpy.histogram(g, bins=numpy.arange(-5, 6, 1))\n",
    "print(bins)\n",
    "print(hist)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [data/SIMAR_gaps.txt](data/SIMAR_gaps.txt) data file gives the wave climate data in the Mediterranean Sea."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Read in the file using ``np.loadtxt``. The data contains bad values, which you can identify by looking at the minimum and maximum values of the array. Use masking to get rid of the bad values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "dir_data = Path('data')\n",
    "data = np.loadtxt(dir_data / 'SIMAR_gaps.txt', skiprows=1)\n",
    "#np.genfromtxt()\n",
    "var = data[:, 4]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's find the bad values by examining the max, min and mean and finding those that do not make sense"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-99.9"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.nanmin(var)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.5"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.nanmax(var)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.4804713804713802"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.nanmean(var)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So the bad values are -99.9. I define a mask for those \"bad\" values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [],
   "source": [
    "mask = var > 0  # mask contains True for \"good values\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Remove the bad values: create a new variable which contains only the \"good\" ones. --> Changing the lenght of the array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "var_good = var[mask]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "var_good"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. Assign NaN for the \"bad\" values. Creating a gap. --> The length of the array stays the same"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [],
   "source": [
    "var[~mask] = np.NaN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Linear Algebra"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "32\n",
      "32\n",
      "[[ 4  5  6]\n",
      " [ 8 10 12]\n",
      " [12 15 18]]\n",
      "(array([1., 2., 3.]), array([[1., 0., 0.],\n",
      "       [0., 1., 0.],\n",
      "       [0., 0., 1.]]))\n",
      "6.0\n",
      "[ 1. -2. -2.]\n"
     ]
    }
   ],
   "source": [
    "import numpy\n",
    "import numpy.linalg as la\n",
    "\n",
    "a = [1,2,3] \n",
    "b = [4,5,6]\n",
    "\n",
    "print(numpy.dot(a,b)) # dot product\n",
    "print(numpy.inner(a,b)) \n",
    "print(numpy.outer(a,b)) \n",
    "\n",
    "i = numpy.diag([1,2,3])\n",
    "print(la.eig(i)) # return eigenvalues and eigenvectors\n",
    "print(la.det(i)) # return determinant\n",
    "\n",
    "# solve linear equations\n",
    "a = [[3,2,-1], [2,-2,4], [-1,0.5,-1]]\n",
    "b = [1,-2,0]\n",
    "print(la.solve(a,b)) "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}