comments on Dask parallelization

notebooks/Parallel.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# Parallelization with Dask\n",
     "\n",
-    "This notebook includes a series of experiments to make multiple samples in parallel using Dark. "
+    "This notebook includes a series of experiments to make multiple samples in parallel using Dask. Many of the simulations carries for this work where carried in parallel and the setup was configures using Dask. We include here the code that was used to create this simulations.  "
    ]
   },
   {
@@ -32,7 +32,9 @@
    "id": "baf2420d-458c-4610-8af1-8969d94f7bbd",
    "metadata": {},
    "source": [
-    "## 1. Dask Setup"
+    "## 1. Dask Setup\n",
+    "\n",
+    "In this case we decided to work with a total of `60` workers, each one of them carrying a different simulation for some given parameters. "
    ]
   },
   {
@@ -2040,6 +2042,14 @@
     "dask_client"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "1d88db41-0133-4e92-b2f6-04f71436c6ba",
+   "metadata": {},
+   "source": [
+    "Check to see if the threads are ready"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 4,
@@ -2058,7 +2068,6 @@
     }
    ],
    "source": [
-    "# Check to see if the threads are ready\n",
     "dview = rc[:]\n",
     "len(dview)"
    ]
@@ -2068,7 +2077,9 @@
    "id": "373fe361-a6da-47ea-a128-3fafa86ad4cf",
    "metadata": {},
    "source": [
-    "## 2. Macro function definition"
+    "## 2. Macro function definition\n",
+    "\n",
+    "We now define a function that is going to execute all the simulation with the desired parameters. In order to this to work in parallel, the class `Params` needs to be redefined inside the function. We further add a hash code to the simulation to keep track of them. "
    ]
   },
   {
@@ -2142,7 +2153,9 @@
    "id": "850a9ed8-136f-451d-9a6a-35d57a96dd1a",
    "metadata": {},
    "source": [
-    "## 3. Parameter space exploration"
+    "## 3. Parameter space exploration\n",
+    "\n",
+    "We now set the different parameter choices used for the different simulations. "
    ]
   },
   {
@@ -2573,7 +2586,9 @@
    "id": "08d0337c-3fd4-430d-b641-aeaca5dbc4f6",
    "metadata": {},
    "source": [
-    "## 4. Run Simulation"
+    "## 4. Run Simulation\n",
+    "\n",
+    "We finally set and run all the simulations. "
    ]
   },
   {
@@ -2616,6 +2631,14 @@
     "progress(res)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "3371279b-f65a-44e7-946a-93cf3baf1ffe",
+   "metadata": {},
+   "source": [
+    "## 5. Save outputs"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -2672,9 +2695,9 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python [conda env:.conda-paleostats]",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
-   "name": "conda-env-.conda-paleostats-py"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -2686,7 +2709,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.2"
+   "version": "3.10.9"
   }
  },
  "nbformat": 4,