04a-Solution-Modeling.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab: Modeling with Dask and Ray\n",
    "\n",
    "To keep things simple, while still giving you a chance to try something hands on, we'll look at \n",
    "\n",
    "* Linear modeling with Dask and a different dataset\n",
    "* Ray RL example using a more powerful algorithm (PPO) than we did earlier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dask and Powerplant Output\n",
    "\n",
    "We'll use the UC Irvine ML repository's Combined Cycle Power Plant Data Set (https://archive.ics.uci.edu/ml/datasets/Combined+Cycle+Power+Plant)\n",
    "\n",
    "This dataset consists of about 10,000 records of measurements relating to peaker power plants.\n",
    "\n",
    "* Temperature (AT) in the range 1.81°C and 37.11°C,\n",
    "* Ambient Pressure (AP) in the range 992.89-1033.30 millibar,\n",
    "* Relative Humidity (RH) in the range 25.56% to 100.16%\n",
    "* Exhaust Vacuum (V) in the range 25.36-81.56 cm Hg\n",
    "* Net hourly electrical energy output (PE) 420.26-495.76 MW\n",
    "\n",
    "We want to model the power output as a function of the other parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import dask.dataframe as ddf\n",
    "\n",
    "df = ddf.read_csv('data/powerplant.csv', sample=False)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = df.PE\n",
    "X = df.drop(columns=['PE'])\n",
    "\n",
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = X.to_dask_array(lengths=True)\n",
    "y = y.to_dask_array(lengths=True)\n",
    "\n",
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dask_ml.model_selection import train_test_split\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1)\n",
    "\n",
    "X_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dask_ml.linear_model import LinearRegression\n",
    "\n",
    "lr = LinearRegression(solver='lbfgs', max_iter=10)\n",
    "lr_model = lr.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_predicted = lr_model.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dask_ml.metrics import mean_squared_error\n",
    "from math import sqrt\n",
    "\n",
    "sqrt(mean_squared_error(y_test, y_predicted))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ray RLlib and PPO\n",
    "\n",
    "PPO of Proximal Policy Optimization is a more powerful (and more complicated) algorithm than the DQN we've looked at.\n",
    "\n",
    "But thanks to Ray's implementations, you can swap it in easily.\n",
    "\n",
    "Note that we import `ppo` from `ray.rllib.agents`\n",
    "\n",
    "By replacing \"DQN\" with \"PPO\" you can quickly get better results.\n",
    "\n",
    ">\n",
    "> Interested in PPO details? Check out this writeup: https://jonathan-hui.medium.com/rl-proximal-policy-optimization-ppo-explained-77f014ec3f12\n",
    ">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import ray\n",
    "import ray.rllib.agents.ppo as ppo\n",
    "\n",
    "ray.shutdown()\n",
    "ray.init()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Specifies the OpenAI Gym environment for CartPole, V1.\n",
    "SELECT_ENV = \"CartPole-v1\"\n",
    "\n",
    "# Number of training runs.\n",
    "N_ITER = 10\n",
    "\n",
    "# default configuration.\n",
    "config = ppo.DEFAULT_CONFIG.copy()\n",
    "\n",
    "# Suppress too many messages.\n",
    "config[\"log_level\"] = \"WARN\"\n",
    "\n",
    "# Use > 1 for more CPU cores, e.g., over a cluster.\n",
    "config['num_workers'] = 2\n",
    "\n",
    "# Describe network\n",
    "config['model']['fcnet_hiddens'] = [40,20]\n",
    "\n",
    "# Don't pin a CPU core to each worker (allows more workers).\n",
    "config['num_cpus_per_worker'] = 0\n",
    "checkpoint_dir = 'checkpoints'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "trainer = ppo.PPOTrainer(config, SELECT_ENV)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fmt = '{:3d},{:8.4f},{:8.4f},{:8.4f}'\n",
    "last_checkpoint = ''\n",
    "for n in range(N_ITER):\n",
    "    result = trainer.train()\n",
    "    min  = result['episode_reward_min']\n",
    "    mean = result['episode_reward_mean']\n",
    "    max  = result['episode_reward_max']\n",
    "    last_checkpoint = trainer.save(checkpoint_dir)\n",
    "    print(fmt.format(n, min, mean, max))\n",
    "print(f'last checkpoint file: {last_checkpoint}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ray.shutdown()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}