Add files via upload

Fix terminal state check in 19.3 and 19.4, fix typo in 19.4

.editorconfig

@@ -0,0 +1,10 @@
+root = true
+
+[*.{js,jsx,ts,tsx,md,mdx,json,cjs,mjs,css}]
+indent_style = space
+indent_size = 4
+end_of_line = lf
+charset = utf-8
+trim_trailing_whitespace = true
+insert_final_newline = true
+max_line_length = 100

.eslintrc.cjs

@@ -0,0 +1,18 @@
+module.exports = {
+    root: true,
+    env: { browser: true, es2020: true, node: true },
+    extends: [
+        "eslint:recommended",
+        "plugin:react/recommended",
+        "plugin:react/jsx-runtime",
+        "plugin:react-hooks/recommended",
+    ],
+    ignorePatterns: ["build", ".eslintrc.cjs"],
+    parserOptions: { ecmaVersion: "latest", sourceType: "module" },
+    settings: { react: { version: "18.2" } },
+    plugins: ["react-refresh"],
+    rules: {
+        "react/jsx-no-target-blank": "off",
+        "react-refresh/only-export-components": ["warn", { allowConstantExport: true }],
+    },
+};

.gitignore

@@ -0,0 +1,30 @@
+# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
+
+# dependencies
+/node_modules
+/.pnp
+.pnp.js
+
+# testing
+/coverage
+
+# production
+/dist
+
+# ENV
+.env.local
+.env.development.local
+.env.test.local
+.env.production.local
+
+# debug
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+
+# IDE
+.idea
+.vscode
+
+# macOS
+.DS_Store

.prettierignore

@@ -0,0 +1,7 @@
+# ignore these directories when formatting the repo
+/Blogs
+/CM20315
+/CM20315_2023
+/Notebooks
+/PDFFigures
+/Slides

.prettierrc.cjs

@@ -0,0 +1,14 @@
+/** @type {import("prettier").Config} */
+const prettierConfig = {
+    trailingComma: "all",
+    tabWidth: 4,
+    useTabs: false,
+    semi: true,
+    singleQuote: false,
+    bracketSpacing: true,
+    printWidth: 100,
+    endOfLine: "lf",
+    plugins: [require.resolve("prettier-plugin-organize-imports")],
+};
+
+module.exports = prettierConfig;

Blogs/BorealisGradientFlow.ipynb

@@ -0,0 +1,401 @@
+{
+  "nbformat": 4,
+  "nbformat_minor": 0,
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "authorship_tag": "ABX9TyP9fLqBQPgcYJB1KXs3Scp/",
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "name": "python3",
+      "display_name": "Python 3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "view-in-github",
+        "colab_type": "text"
+      },
+      "source": [
+        "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/Blogs/BorealisGradientFlow.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "# Gradient flow\n",
+        "\n",
+        "This notebook replicates some of the results in the Borealis AI [blog](https://www.borealisai.com/research-blogs/gradient-flow/) on gradient flow.  \n"
+      ],
+      "metadata": {
+        "id": "ucrRRJ4dq8_d"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Import relevant libraries\n",
+        "import numpy as np\n",
+        "import matplotlib.pyplot as plt\n",
+        "from scipy.linalg import expm\n",
+        "from matplotlib import cm\n",
+        "from matplotlib.colors import ListedColormap"
+      ],
+      "metadata": {
+        "id": "_IQFHZEMZE8T"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "Create the three data points that are used to train the linear model in the blog.  Each input point is a column in $\\mathbf{X}$ and consists of the $x$ position in the plot and the value 1, which is used to allow the model to fit bias terms neatly."
+      ],
+      "metadata": {
+        "id": "NwgUP3MSriiJ"
+      }
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "cJNZ2VIcYsD8"
+      },
+      "outputs": [],
+      "source": [
+        "X = np.array([[0.2, 0.4, 0.8],[1,1,1]])\n",
+        "y = np.array([[-0.1],[0.15],[0.3]])\n",
+        "D = X.shape[0]\n",
+        "I  = X.shape[1]\n",
+        "\n",
+        "print(\"X=\\n\",X)\n",
+        "print(\"y=\\n\",y)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Draw the three data points\n",
+        "fig, ax  = plt.subplots()\n",
+        "ax.plot(X[0:1,:],y.T,'ro')\n",
+        "ax.set_xlim([0,1]); ax.set_ylim([-0.5,0.5])\n",
+        "ax.set_xlabel('x'); ax.set_ylabel('y')\n",
+        "plt.show()"
+      ],
+      "metadata": {
+        "id": "FpFlD4nUZDRt"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "Compute the evolution of the residuals, loss, and parameters as a function of time."
+      ],
+      "metadata": {
+        "id": "H2LBR1DasQej"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Discretized time to evaluate quantities at\n",
+        "t_all = np.arange(0,20,0.01)\n",
+        "nT = t_all.shape[0]\n",
+        "\n",
+        "# Initial parameters, and initial function output at training points\n",
+        "phi_0 = np.array([[-0.05],[-0.4]])\n",
+        "f_0 = X.T @ phi_0\n",
+        "\n",
+        "# Precompute pseudoinverse term (not a very sensible numerical implementation, but it works...)\n",
+        "XXTInvX = np.linalg.inv(X@X.T)@X\n",
+        "\n",
+        "# Create arrays to hold function at data points over time, residual over time, parameters over time\n",
+        "f_all = np.zeros((I,nT))\n",
+        "f_minus_y_all = np.zeros((I,nT))\n",
+        "phi_t_all = np.zeros((D,nT))\n",
+        "\n",
+        "# For each time, compute function, residual, and parameters at each time.\n",
+        "for t in range(len(t_all)):\n",
+        "  f = y + expm(-X.T@X * t_all[t]) @ (f_0-y)\n",
+        "  f_all[:,t:t+1] = f\n",
+        "  f_minus_y_all[:,t:t+1] = f-y\n",
+        "  phi_t_all[:,t:t+1] = phi_0 - XXTInvX @ (np.identity(3)-expm(-X.T@X * t_all[t])) @ (f_0-y)"
+      ],
+      "metadata": {
+        "id": "wfF_oTS5Z4Wi"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "Plot the results that were calculated in the previous cell"
+      ],
+      "metadata": {
+        "id": "9jSjOOFutJUE"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Plot function at data points\n",
+        "fig, ax  = plt.subplots()\n",
+        "ax.plot(t_all,np.squeeze(f_all[0,:]),'r-', label='$f[x_{0},\\phi]$')\n",
+        "ax.plot(t_all,np.squeeze(f_all[1,:]),'g-', label='$f[x_{1},\\phi]$')\n",
+        "ax.plot(t_all,np.squeeze(f_all[2,:]),'b-', label='$f[x_{2},\\phi]$')\n",
+        "ax.set_xlim([0,np.max(t_all)]); ax.set_ylim([-0.5,0.5])\n",
+        "ax.set_xlabel('t'); ax.set_ylabel('f')\n",
+        "plt.legend(loc=\"lower right\")\n",
+        "plt.show()\n",
+        "\n",
+        "# Plot residual\n",
+        "fig, ax  = plt.subplots()\n",
+        "ax.plot(t_all,np.squeeze(f_minus_y_all[0,:]),'r-', label='$f[x_{0},\\phi]-y_{0}$')\n",
+        "ax.plot(t_all,np.squeeze(f_minus_y_all[1,:]),'g-', label='$f[x_{1},\\phi]-y_{1}$')\n",
+        "ax.plot(t_all,np.squeeze(f_minus_y_all[2,:]),'b-', label='$f[x_{2},\\phi]-y_{2}$')\n",
+        "ax.set_xlim([0,np.max(t_all)]); ax.set_ylim([-0.5,0.5])\n",
+        "ax.set_xlabel('t'); ax.set_ylabel('f-y')\n",
+        "plt.legend(loc=\"lower right\")\n",
+        "plt.show()\n",
+        "\n",
+        "# Plot loss (sum of residuals)\n",
+        "fig, ax  = plt.subplots()\n",
+        "square_error = 0.5 * np.sum(f_minus_y_all * f_minus_y_all, axis=0)\n",
+        "ax.plot(t_all, square_error,'k-')\n",
+        "ax.set_xlim([0,np.max(t_all)]); ax.set_ylim([-0.0,0.25])\n",
+        "ax.set_xlabel('t'); ax.set_ylabel('Loss')\n",
+        "plt.show()\n",
+        "\n",
+        "# Plot parameters\n",
+        "fig, ax  = plt.subplots()\n",
+        "ax.plot(t_all, np.squeeze(phi_t_all[0,:]),'c-',label='$\\phi_{0}$')\n",
+        "ax.plot(t_all, np.squeeze(phi_t_all[1,:]),'m-',label='$\\phi_{1}$')\n",
+        "ax.set_xlim([0,np.max(t_all)]); ax.set_ylim([-1,1])\n",
+        "ax.set_xlabel('t'); ax.set_ylabel('$\\phi$')\n",
+        "plt.legend(loc=\"lower right\")\n",
+        "plt.show()"
+      ],
+      "metadata": {
+        "id": "G9IwgwKltHz5"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "Define the model and the loss function"
+      ],
+      "metadata": {
+        "id": "N6VaUq2swa8D"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Model is just a straight line with intercept phi[0] and slope phi[1]\n",
+        "def model(phi,x):\n",
+        "  y_pred = phi[0]+phi[1] * x\n",
+        "  return y_pred\n",
+        "\n",
+        "# Loss function is 0.5 times sum of squares of residuals for training data\n",
+        "def compute_loss(data_x, data_y, model, phi):\n",
+        "  pred_y = model(phi, data_x)\n",
+        "  loss = 0.5 * np.sum((pred_y-data_y)*(pred_y-data_y))\n",
+        "  return loss"
+      ],
+      "metadata": {
+        "id": "LGHEVUWWiB4f"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "Draw the loss function"
+      ],
+      "metadata": {
+        "id": "hr3hs7pKwo0g"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "def draw_loss_function(compute_loss, X, y,  model, phi_iters):\n",
+        "  # Define pretty colormap\n",
+        "  my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+        "  my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+        "  r = np.floor(my_colormap_vals_dec/(256*256))\n",
+        "  g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+        "  b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+        "  my_colormap = ListedColormap(np.vstack((r,g,b)).transpose()/255.0)\n",
+        "\n",
+        "  # Make grid of intercept/slope values to plot\n",
+        "  intercepts_mesh, slopes_mesh = np.meshgrid(np.arange(-1.0,1.0,0.005), np.arange(-1.0,1.0,0.005))\n",
+        "  loss_mesh = np.zeros_like(slopes_mesh)\n",
+        "  # Compute loss for every set of parameters\n",
+        "  for idslope, slope in np.ndenumerate(slopes_mesh):\n",
+        "     loss_mesh[idslope] = compute_loss(X, y, model, np.array([[intercepts_mesh[idslope]], [slope]]))\n",
+        "\n",
+        "  fig,ax = plt.subplots()\n",
+        "  fig.set_size_inches(8,8)\n",
+        "  ax.contourf(intercepts_mesh,slopes_mesh,loss_mesh,256,cmap=my_colormap)\n",
+        "  ax.contour(intercepts_mesh,slopes_mesh,loss_mesh,40,colors=['#80808080'])\n",
+        "  ax.set_ylim([1,-1]); ax.set_xlim([-1,1])\n",
+        "\n",
+        "  ax.plot(phi_iters[1,:], phi_iters[0,:],'g-')\n",
+        "  ax.set_xlabel('Intercept'); ax.set_ylabel('Slope')\n",
+        "  plt.show()"
+      ],
+      "metadata": {
+        "id": "UCxa3tZ8a9kz"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "draw_loss_function(compute_loss, X[0:1,:], y.T, model, phi_t_all)"
+      ],
+      "metadata": {
+        "id": "pXLLBaSaiI2A"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "Draw the evolution of the function"
+      ],
+      "metadata": {
+        "id": "ZsremHW-xFi5"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "fig, ax  = plt.subplots()\n",
+        "ax.plot(X[0:1,:],y.T,'ro')\n",
+        "x_vals = np.arange(0,1,0.001)\n",
+        "ax.plot(x_vals, phi_t_all[0,0]*x_vals + phi_t_all[1,0],'r-', label='t=0.00')\n",
+        "ax.plot(x_vals, phi_t_all[0,10]*x_vals + phi_t_all[1,10],'g-', label='t=0.10')\n",
+        "ax.plot(x_vals, phi_t_all[0,30]*x_vals + phi_t_all[1,30],'b-', label='t=0.30')\n",
+        "ax.plot(x_vals, phi_t_all[0,200]*x_vals + phi_t_all[1,200],'c-', label='t=2.00')\n",
+        "ax.plot(x_vals, phi_t_all[0,1999]*x_vals + phi_t_all[1,1999],'y-', label='t=20.0')\n",
+        "ax.set_xlim([0,1]); ax.set_ylim([-0.5,0.5])\n",
+        "ax.set_xlabel('x'); ax.set_ylabel('y')\n",
+        "plt.legend(loc=\"upper left\")\n",
+        "plt.show()"
+      ],
+      "metadata": {
+        "id": "cv9ZrUoRkuhI"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Compute MAP and ML solutions\n",
+        "MLParams = np.linalg.inv(X@X.T)@X@y\n",
+        "sigma_sq_p = 3.0\n",
+        "sigma_sq = 0.05\n",
+        "MAPParams = np.linalg.inv(X@X.T+np.identity(X.shape[0])*sigma_sq/sigma_sq_p)@X@y"
+      ],
+      "metadata": {
+        "id": "OU9oegSOof-o"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "Finally, we predict both the mean and the uncertainty in the fitted model as a function of time"
+      ],
+      "metadata": {
+        "id": "Ul__XvOgyYSA"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Define x positions to make predictions (appending a 1 to each column)\n",
+        "x_predict = np.arange(0,1,0.01)[None,:]\n",
+        "x_predict = np.concatenate((x_predict,np.ones_like(x_predict)))\n",
+        "nX = x_predict.shape[1]\n",
+        "\n",
+        "# Create variables to store evolution of mean and variance of prediction over time\n",
+        "predict_mean_all = np.zeros((nT,nX))\n",
+        "predict_var_all = np.zeros((nT,nX))\n",
+        "\n",
+        "# Initial covariance\n",
+        "sigma_sq_p = 2.0\n",
+        "cov_init = sigma_sq_p * np.identity(2)\n",
+        "\n",
+        "# Run through each time computing a and b and hence mean and variance of prediction\n",
+        "for t in range(len(t_all)):\n",
+        "  a = x_predict.T @(XXTInvX @ (np.identity(3)-expm(-X.T@X * t_all[t])) @ y)\n",
+        "  b = x_predict.T -x_predict.T@XXTInvX @ (np.identity(3)-expm(-X.T@X * t_all[t])) @ X.T\n",
+        "  predict_mean_all[t:t+1,:] = a.T\n",
+        "  predict_cov  = b@ cov_init @b.T\n",
+        "  # We just want the diagonal of the covariance to plot the uncertainty\n",
+        "  predict_var_all[t:t+1,:] = np.reshape(np.diag(predict_cov),(1,nX))"
+      ],
+      "metadata": {
+        "id": "aMPADCuByKWr"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "Plot the mean and variance at various times"
+      ],
+      "metadata": {
+        "id": "PZTj93KK7QH6"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "def plot_mean_var(X,y,x_predict, predict_mean_all, predict_var_all, this_t, sigma_sq = 0.00001):\n",
+        "  fig, ax  = plt.subplots()\n",
+        "  ax.plot(X[0:1,:],y.T,'ro')\n",
+        "  ax.plot(x_predict[0:1,:].T, predict_mean_all[this_t:this_t+1,:].T,'r-')\n",
+        "  lower = np.squeeze(predict_mean_all[this_t:this_t+1,:].T-np.sqrt(predict_var_all[this_t:this_t+1,:].T+np.sqrt(sigma_sq)))\n",
+        "  upper = np.squeeze(predict_mean_all[this_t:this_t+1,:].T+np.sqrt(predict_var_all[this_t:this_t+1,:].T+np.sqrt(sigma_sq)))\n",
+        "  ax.fill_between(np.squeeze(x_predict[0:1,:]), lower, upper, color='lightgray')\n",
+        "  ax.set_xlim([0,1]); ax.set_ylim([-0.5,0.5])\n",
+        "  ax.set_xlabel('x'); ax.set_ylabel('y')\n",
+        "  plt.show()\n",
+        "\n",
+        "plot_mean_var(X,y,x_predict, predict_mean_all, predict_var_all, this_t=0)\n",
+        "plot_mean_var(X,y,x_predict, predict_mean_all, predict_var_all, this_t=40)\n",
+        "plot_mean_var(X,y,x_predict, predict_mean_all, predict_var_all, this_t=80)\n",
+        "plot_mean_var(X,y,x_predict, predict_mean_all, predict_var_all, this_t=200)\n",
+        "plot_mean_var(X,y,x_predict, predict_mean_all, predict_var_all, this_t=500)\n",
+        "plot_mean_var(X,y,x_predict, predict_mean_all, predict_var_all, this_t=1000)"
+      ],
+      "metadata": {
+        "id": "bYAFxgB880-v"
+      },
+      "execution_count": null,
+      "outputs": []
+    }
+  ]
+}

Blogs/BorealisODENumerical.ipynb

@@ -0,0 +1,432 @@
+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "view-in-github",
+        "colab_type": "text"
+      },
+      "source": [
+        "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/Blogs/BorealisODENumerical.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "JXsO7ce7oqeq"
+      },
+      "source": [
+        "# Numerical methods for ODEs\n",
+        "\n",
+        "This blog contains code that accompanies the RBC Borealis blog on numerical methods for ODEs. Contact udlbookmail@gmail.com if you find any problems."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "AnvAKtP_oqes"
+      },
+      "source": [
+        "Import relevant libraries"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "UF-gJyZggyrl"
+      },
+      "outputs": [],
+      "source": [
+        "import numpy as np\n",
+        "import matplotlib.pyplot as plt"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "szWLVrSSoqet"
+      },
+      "source": [
+        "Define the ODE that we will be experimenting with."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "NkrGZLL6iM3P"
+      },
+      "outputs": [],
+      "source": [
+        "# The ODE that we will experiment with\n",
+        "def ode_lin_homog(t,x):\n",
+        "  return 0.5 * x ;\n",
+        "\n",
+        "# The derivative of the ODE function with respect to x (needed for Taylor's method)\n",
+        "def ode_lin_homog_deriv_x(t,x):\n",
+        "  return 0.5 ;\n",
+        "\n",
+        "# The derivative of the ODE function with respect to t (needed for Taylor's method)\n",
+        "def ode_lin_homog_deriv_t(t,x):\n",
+        "  return 0.0 ;\n",
+        "\n",
+        "# The closed form solution (so we can measure the error)\n",
+        "def ode_lin_homog_soln(t,C=0.5):\n",
+        "  return C * np.exp(0.5 * t) ;"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "In1C9wZkoqet"
+      },
+      "source": [
+        "This is a generic method that runs the numerical methods. It takes the initial conditions ($t_0$, $x_0$), the final time $t_1$ and the step size $h$.  It also takes the ODE function itself and its derivatives (only used for Taylor's method).  Finally, the parameter \"step_function\" is the method used to update (e.g., Euler's methods, Runge-Kutte 4-step)."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "VZfZDJAfmyrf"
+      },
+      "outputs": [],
+      "source": [
+        "def run_numerical(x_0, t_0, t_1, h, ode_func, ode_func_deriv_x, ode_func_deriv_t, ode_soln, step_function):\n",
+        "  x = [x_0]\n",
+        "  t = [t_0]\n",
+        "  while (t[-1] <= t_1):\n",
+        "    x = x+[step_function(x[-1],t[-1],h, ode_func, ode_func_deriv_x, ode_func_deriv_t)]\n",
+        "    t = t + [t[-1]+h]\n",
+        "\n",
+        "  # Returns x,y plot plus total numerical error at last point.\n",
+        "  return t, x, np.abs(ode_soln(t[-1])-x[-1])"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "Vfkc3-_7oqet"
+      },
+      "source": [
+        "Run the numerical method with step sizes of 2.0, 1.0, 0.5, 0.25, 0.125, 0.0675 and plot the results"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "1tyGbMZhoqeu"
+      },
+      "outputs": [],
+      "source": [
+        "def run_and_plot(ode, ode_deriv_x, ode_deriv_t, ode_solution, step_function):\n",
+        "    # Specify the grid of points to draw the ODE\n",
+        "    t = np.arange(0.04, 4.0, 0.2)\n",
+        "    x = np.arange(0.04, 4.0, 0.2)\n",
+        "    T, X = np.meshgrid(t,x)\n",
+        "\n",
+        "    # ODE equation at these grid points (used to draw quiver-plot)\n",
+        "    dx = ode(T,X)\n",
+        "    dt = np.ones(dx.shape)\n",
+        "\n",
+        "    # The ground truth solution\n",
+        "    t2= np.arange(0,10,0.1)\n",
+        "    x2 = ode_solution(t2)\n",
+        "\n",
+        "    #####################################x_0, t_0, t_1, h #################################################\n",
+        "    t_sim1,x_sim1,error1 = run_numerical(0.5, 0.0, 4.0, 2.0000, ode, ode_deriv_x, ode_deriv_t, ode_solution, step_function)\n",
+        "    t_sim2,x_sim2,error2 = run_numerical(0.5, 0.0, 4.0, 1.0000, ode, ode_deriv_x, ode_deriv_t, ode_solution, step_function)\n",
+        "    t_sim3,x_sim3,error3 = run_numerical(0.5, 0.0, 4.0, 0.5000, ode, ode_deriv_x, ode_deriv_t, ode_solution, step_function)\n",
+        "    t_sim4,x_sim4,error4 = run_numerical(0.5, 0.0, 4.0, 0.2500, ode, ode_deriv_x, ode_deriv_t, ode_solution, step_function)\n",
+        "    t_sim5,x_sim5,error5 = run_numerical(0.5, 0.0, 4.0, 0.1250, ode, ode_deriv_x, ode_deriv_t, ode_solution, step_function)\n",
+        "    t_sim6,x_sim6,error6 = run_numerical(0.5, 0.0, 4.0, 0.0675, ode, ode_deriv_x, ode_deriv_t, ode_solution, step_function)\n",
+        "\n",
+        "    # Plot the ODE and ground truth solution\n",
+        "    fig,ax = plt.subplots()\n",
+        "    ax.quiver(T,X,dt,dx, scale=35.0)\n",
+        "    ax.plot(t2,x2,'r-')\n",
+        "\n",
+        "    # Plot the numerical approximations\n",
+        "    ax.plot(t_sim1,x_sim1,'.-',markeredgecolor='#773c23ff',markerfacecolor='#d18362', color='#d18362', markersize=10)\n",
+        "    ax.plot(t_sim2,x_sim2,'.-',markeredgecolor='#773c23ff',markerfacecolor='#d18362', color='#d18362', markersize=10)\n",
+        "    ax.plot(t_sim3,x_sim3,'.-',markeredgecolor='#773c23ff',markerfacecolor='#d18362', color='#d18362', markersize=10)\n",
+        "    ax.plot(t_sim4,x_sim4,'.-',markeredgecolor='#773c23ff',markerfacecolor='#d18362', color='#d18362', markersize=10)\n",
+        "    ax.plot(t_sim5,x_sim5,'.-',markeredgecolor='#773c23ff',markerfacecolor='#d18362', color='#d18362', markersize=10)\n",
+        "    ax.plot(t_sim6,x_sim6,'.-',markeredgecolor='#773c23ff',markerfacecolor='#d18362', color='#d18362', markersize=10)\n",
+        "\n",
+        "    ax.set_aspect('equal')\n",
+        "    ax.set_xlim(0,4)\n",
+        "    ax.set_ylim(0,4)\n",
+        "\n",
+        "    plt.show()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "JYrq8QIwvOIy"
+      },
+      "source": [
+        "# Euler Method\n",
+        "\n",
+        "Define the Euler method and set up functions for plotting."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "N73xMnCukVVX"
+      },
+      "outputs": [],
+      "source": [
+        "def euler_step(x_0, t_0, h, ode_func, ode_func_deriv_x=None, ode_func_deriv_t=None):\n",
+        "  return x_0 + h * ode_func(t_0, x_0) ;"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "4B1_PGEcsZ9H"
+      },
+      "outputs": [],
+      "source": [
+        "run_and_plot(ode_lin_homog, None, None, ode_lin_homog_soln, euler_step)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "FfwNihtkvJeX"
+      },
+      "source": [
+        "# Heun's Method"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "srHfNDcDxI1o"
+      },
+      "outputs": [],
+      "source": [
+        "def heun_step(x_0, t_0, h, ode_func, ode_func_deriv_x=None, ode_func_deriv_t=None):\n",
+        "  f_x0_t0 = ode_func(t_0, x_0)\n",
+        "  return x_0 + h/2 * ( f_x0_t0 + ode_func(t_0+h, x_0+h*f_x0_t0)) ;"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "WOApHz9xoqev"
+      },
+      "outputs": [],
+      "source": [
+        "run_and_plot(ode_lin_homog, None, None, ode_lin_homog_soln, heun_step)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "0XSzzFDIvRhm"
+      },
+      "source": [
+        "# Modified Euler method"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "fSXprgVJ5Yep"
+      },
+      "outputs": [],
+      "source": [
+        "def modified_euler_step(x_0, t_0, h, ode_func, ode_func_deriv_x=None, ode_func_deriv_t=None):\n",
+        "  f_x0_t0 = ode_func(t_0, x_0)\n",
+        "  return x_0 + h * ode_func(t_0+h/2, x_0+ h * f_x0_t0/2) ;"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "8LKSrCD2oqev"
+      },
+      "outputs": [],
+      "source": [
+        "run_and_plot(ode_lin_homog, None, None, ode_lin_homog_soln, modified_euler_step)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "yp8ZBpwooqev"
+      },
+      "source": [
+        "# Second order Taylor's method"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "NtBBgzWLoqev"
+      },
+      "outputs": [],
+      "source": [
+        "def taylor_2nd_order(x_0, t_0, h, ode_func, ode_func_deriv_x, ode_func_deriv_t):\n",
+        "    f1 = ode_func(t_0, x_0)\n",
+        "    return x_0 + h * ode_func(t_0, x_0) + (h*h/2) * (ode_func_deriv_x(t_0,x_0) * ode_func(t_0, x_0) + ode_func_deriv_t(t_0, x_0))"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ioeeIohUoqev"
+      },
+      "outputs": [],
+      "source": [
+        "run_and_plot(ode_lin_homog, ode_lin_homog_deriv_x, ode_lin_homog_deriv_t, ode_lin_homog_soln, taylor_2nd_order)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "WcuhV5lL1zAJ"
+      },
+      "source": [
+        "# Fourth Order Runge Kutta"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "0NZN81Bpwu56"
+      },
+      "outputs": [],
+      "source": [
+        "def runge_kutta_4_step(x_0, t_0, h, ode_func, ode_func_deriv_x=None, ode_func_deriv_t=None):\n",
+        "    f1 = ode_func(t_0, x_0)\n",
+        "    f2 = ode_func(t_0+h/2,x_0+f1 * h/2)\n",
+        "    f3 = ode_func(t_0+h/2,x_0+f2 * h/2)\n",
+        "    f4 = ode_func(t_0+h, x_0+ f3*h)\n",
+        "    return x_0 + (h/6) * (f1 + 2*f2 + 2*f3+f4)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "K-OxE9E6oqew"
+      },
+      "outputs": [],
+      "source": [
+        "run_and_plot(ode_lin_homog, None, None, ode_lin_homog_soln, runge_kutta_4_step)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "7JifxBhhoqew"
+      },
+      "source": [
+        "# Plot the error as a function of step size"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ZoEpmlCfsi9P"
+      },
+      "outputs": [],
+      "source": [
+        "# Run systematically with a number of different step sizes and store errors for each\n",
+        "def get_errors(ode, ode_deriv_x, ode_deriv_t, ode_solution, step_function):\n",
+        "    # Choose the step size h to divide the plotting interval into 1,2,4,8... segments.\n",
+        "    # The plots in the article add a few more smaller step sizes, but this takes a while to compute.\n",
+        "    # Add them back in if you want the full plot.\n",
+        "    all_h = (1./np.array([1,2,4,8,16,32,64,128,256,512,1024,2048,4096])).tolist()\n",
+        "    all_err = []\n",
+        "\n",
+        "    for i in range(len(all_h)):\n",
+        "        t_sim,x_sim,err = run_numerical(0.5, 0.0, 4.0, all_h[i], ode, ode_deriv_x, ode_deriv_t, ode_solution, step_function)\n",
+        "        all_err = all_err + [err]\n",
+        "\n",
+        "    return all_h, all_err"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "X0O0KK47xF28"
+      },
+      "outputs": [],
+      "source": [
+        "# Plot the errors\n",
+        "all_h, all_err_euler = get_errors(ode_lin_homog, ode_lin_homog_deriv_x, ode_lin_homog_deriv_t, ode_lin_homog_soln, euler_step)\n",
+        "all_h, all_err_heun = get_errors(ode_lin_homog, ode_lin_homog_deriv_x, ode_lin_homog_deriv_t, ode_lin_homog_soln, heun_step)\n",
+        "all_h, all_err_mod_euler = get_errors(ode_lin_homog, ode_lin_homog_deriv_x, ode_lin_homog_deriv_t, ode_lin_homog_soln, modified_euler_step)\n",
+        "all_h, all_err_taylor = get_errors(ode_lin_homog, ode_lin_homog_deriv_x, ode_lin_homog_deriv_t, ode_lin_homog_soln, taylor_2nd_order)\n",
+        "all_h, all_err_rk = get_errors(ode_lin_homog, ode_lin_homog_deriv_x, ode_lin_homog_deriv_t, ode_lin_homog_soln, runge_kutta_4_step)\n",
+        "\n",
+        "\n",
+        "fig, ax = plt.subplots()\n",
+        "ax.loglog(all_h, all_err_euler,'ro-')\n",
+        "ax.loglog(all_h, all_err_heun,'bo-')\n",
+        "ax.loglog(all_h, all_err_mod_euler,'go-')\n",
+        "ax.loglog(all_h, all_err_taylor,'co-')\n",
+        "ax.loglog(all_h, all_err_rk,'mo-')\n",
+        "ax.set_ylim(1e-13,1e1)\n",
+        "ax.set_xlim(1e-6,1e1)\n",
+        "ax.set_aspect(0.5)\n",
+        "ax.set_xlabel('Step size, $h$')\n",
+        "ax.set_ylabel('Error')\n",
+        "plt.show()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "BttOqpeo9MsJ"
+      },
+      "source": [
+        "Note that for this ODE, the Heun, Modified Euler and Taylor methods provide EXACTLY the same updates, and so the error curves for all three are identical (subject to difference is numerical rounding errors).  This is not in general the case, although the general trend would be the same for each."
+      ]
+    }
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3 (ipykernel)",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.9.10"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}

CM20315/CM20315_Convolution_II.ipynb

@@ -105,7 +105,7 @@
       "cell_type": "code",
       "source": [
         "\n",
-        "# TODO Create a model with the folowing layers\n",
+        "# TODO Create a model with the following layers\n",
         "# 1. Convolutional layer, (input=length 40 and 1 channel, kernel size 3x3, stride 2, padding=\"valid\", 15 output channels ) \n",
         "# 2. ReLU\n",
         "# 3. Convolutional layer, (input=length 19 and 15 channels, kernel size 3x3, stride 2, padding=\"valid\", 15 output channels )\n",
@@ -120,7 +120,7 @@
         "# https://pytorch.org/docs/1.13/generated/torch.nn.Linear.html?highlight=linear#torch.nn.Linear\n",
         "\n",
         "# Replace the following function which just runs a standard fully connected network\n",
-        "# The flatten at the beginning is becuase we are passing in the data in a slightly different format.\n",
+        "# The flatten at the beginning is because we are passing in the data in a slightly different format.\n",
         "model = nn.Sequential(\n",
         "nn.Flatten(),\n",
         "nn.Linear(40, 100),\n",

CM20315/CM20315_Convolution_III.ipynb

@@ -148,7 +148,7 @@
         "# 8. A flattening operation\n",
         "# 9. A fully connected layer mapping from (whatever dimensions we are at-- find out using .shape) to 50 \n",
         "# 10. A ReLU\n",
-        "# 11. A fully connected layer mappiing from 50 to 10 dimensions\n",
+        "# 11. A fully connected layer mapping from 50 to 10 dimensions\n",
         "# 12. A softmax function.\n",
         "\n",
         "# Replace this class which implements a minimal network (which still does okay)\n",

CM20315/CM20315_Coursework_IV.ipynb

@@ -128,7 +128,7 @@
         "\n",
         "In part (b) of the practical we calculate the volume of a hypersphere of radius 0.5 (i.e., of diameter 1) as a function of the radius.  You will find that the volume decreases to almost nothing in high dimensions.  All of the volume is in the corners of the unit hypercube (which always has volume 1). Double weird.\n",
         "\n",
-        "Note that you you can check your answer by doing the calculation for 2D using the standard formula for the area of a circle and making sure it matches."
+        "Note that you can check your answer by doing the calculation for 2D using the standard formula for the area of a circle and making sure it matches."
       ],
       "metadata": {
         "id": "b2FYKV1SL4Z7"

CM20315/CM20315_Gradients_II.ipynb

@@ -32,7 +32,7 @@
       "source": [
         "# Gradients II: Backpropagation algorithm\n",
         "\n",
-        "In this practical, we'll investigate the backpropagation algoritithm.  This computes the gradients of the loss with respect to all of the parameters (weights and biases) in the network.  We'll use these gradients when we run stochastic gradient descent."
+        "In this practical, we'll investigate the backpropagation algorithm.  This computes the gradients of the loss with respect to all of the parameters (weights and biases) in the network.  We'll use these gradients when we run stochastic gradient descent."
       ],
       "metadata": {
         "id": "L6chybAVFJW2"
@@ -53,7 +53,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "First let's define a neural network.  We'll just choose the weights and biaes randomly for now"
+        "First let's define a neural network.  We'll just choose the weights and biases randomly for now"
       ],
       "metadata": {
         "id": "nnUoI0m6GyjC"
@@ -178,7 +178,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Now let's define a loss function.  We'll just use the least squaures loss function. We'll also write a function to compute dloss_doutpu"
+        "Now let's define a loss function.  We'll just use the least squares loss function. We'll also write a function to compute dloss_doutpu"
       ],
       "metadata": {
         "id": "SxVTKp3IcoBF"

CM20315/CM20315_Gradients_III.ipynb

@@ -53,7 +53,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "First let's define a neural network.  We'll just choose the weights and biaes randomly for now"
+        "First let's define a neural network.  We'll just choose the weights and biases randomly for now"
       ],
       "metadata": {
         "id": "nnUoI0m6GyjC"
@@ -204,7 +204,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Now let's define a loss function.  We'll just use the least squaures loss function. We'll also write a function to compute dloss_doutput\n"
+        "Now let's define a loss function.  We'll just use the least squares loss function. We'll also write a function to compute dloss_doutput\n"
       ],
       "metadata": {
         "id": "SxVTKp3IcoBF"

CM20315/CM20315_Intro.ipynb

@@ -176,7 +176,7 @@
         "# Color represents y value (brighter = higher value)\n",
         "# Black = -10 or less, White = +10 or more\n",
         "# 0 = mid orange\n",
-        "# Lines are conoturs where value is equal\n",
+        "# Lines are contours where value is equal\n",
         "draw_2D_function(x1,x2,y)\n",
         "\n",
         "# TODO\n",

CM20315/CM20315_Intro_Answers.ipynb

@@ -215,7 +215,7 @@
         "# Color represents y value (brighter = higher value)\n",
         "# Black = -10 or less, White = +10 or more\n",
         "# 0 = mid orange\n",
-        "# Lines are conoturs where value is equal\n",
+        "# Lines are contours where value is equal\n",
         "draw_2D_function(x1,x2,y)\n",
         "\n",
         "# TODO\n",

CM20315/CM20315_Loss.ipynb

@@ -36,7 +36,7 @@
         "\n",
         "We'll compute loss functions for maximum likelihood, minimum negative log likelihood, and least squares and show that they all imply that we should use the same parameter values\n",
         "\n",
-        "In part II, we'll investigate binary classification (where the output data is 0 or 1).  This will be based on the Bernouilli distribution\n",
+        "In part II, we'll investigate binary classification (where the output data is 0 or 1).  This will be based on the Bernoulli distribution\n",
         "\n",
         "In part III we'll investigate multiclass classification (where the output data is 0,1, or, 2).  This will be based on the categorical distribution."
       ],
@@ -178,7 +178,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "The blue line i sthe mean prediction of the model and the gray area represents plus/minus two standardard deviations.  This model fits okay, but could be improved. Let's compute the loss.  We'll compute the  the least squares error, the likelihood, the negative log likelihood."
+        "The blue line is the mean prediction of the model and the gray area represents plus/minus two standard deviations.  This model fits okay, but could be improved. Let's compute the loss.  We'll compute the  the least squares error, the likelihood, the negative log likelihood."
       ],
       "metadata": {
         "id": "MvVX6tl9AEXF"
@@ -276,7 +276,7 @@
         "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
         "# Use our neural network to predict the mean of the Gaussian\n",
         "mu_pred = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
-        "# Set the standard devation to something reasonable\n",
+        "# Set the standard deviation to something reasonable\n",
         "sigma = 0.2\n",
         "# Compute the likelihood\n",
         "likelihood = compute_likelihood(y_train, mu_pred, sigma)\n",
@@ -292,7 +292,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "You can see that this gives a very small answer, even for this small 1D dataset, and with the model fitting quite well.  This is because it is the product of sveral probabilities, which are all quite small themselves.\n",
+        "You can see that this gives a very small answer, even for this small 1D dataset, and with the model fitting quite well.  This is because it is the product of several probabilities, which are all quite small themselves.\n",
         "This will get out of hand pretty quickly with real datasets -- the likelihood will get so small that we can't represent it with normal finite-precision math\n",
         "\n",
         "This is why we use negative log likelihood"
@@ -326,7 +326,7 @@
         "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
         "# Use our neural network to predict the mean of the Gaussian\n",
         "mu_pred = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
-        "# Set the standard devation to something reasonable\n",
+        "# Set the standard deviation to something reasonable\n",
         "sigma = 0.2\n",
         "# Compute the log likelihood\n",
         "nll = compute_negative_log_likelihood(y_train, mu_pred, sigma)\n",
@@ -397,7 +397,7 @@
       "source": [
         "# Define a range of values for the parameter\n",
         "beta_1_vals = np.arange(0,1.0,0.01)\n",
-        "# Create some arrays to store the likelihoods, negative log likehoos and sum of squares\n",
+        "# Create some arrays to store the likelihoods, negative log likelihoods and sum of squares\n",
         "likelihoods = np.zeros_like(beta_1_vals)\n",
         "nlls = np.zeros_like(beta_1_vals)\n",
         "sum_squares = np.zeros_like(beta_1_vals)\n",
@@ -482,7 +482,7 @@
       "source": [
         "# Define a range of values for the parameter\n",
         "sigma_vals = np.arange(0.1,0.5,0.005)\n",
-        "# Create some arrays to store the likelihoods, negative log likehoos and sum of squares\n",
+        "# Create some arrays to store the likelihoods, negative log likelihoods and sum of squares\n",
         "likelihoods = np.zeros_like(sigma_vals)\n",
         "nlls = np.zeros_like(sigma_vals)\n",
         "sum_squares = np.zeros_like(sigma_vals)\n",

CM20315/CM20315_Loss_II.ipynb

@@ -34,7 +34,7 @@
         "\n",
         "This practical investigates loss functions.  In part I we investigated univariate regression (where the output data $y$ is continuous.  Our formulation was based on the normal/Gaussian distribution.\n",
         "\n",
-        "In this notebook, we investigate binary classification (where the output data is 0 or 1).  This will be based on the Bernouilli distribution\n",
+        "In this notebook, we investigate binary classification (where the output data is 0 or 1).  This will be based on the Bernoulli distribution\n",
         "\n",
         "In part III we'll investigate multiclass classification (where the outputs data can take multiple values 1,... K.\n",
         "\n",
@@ -199,7 +199,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "The left is model output and the right is the model output after the sigmoid has been applied, so it now lies in the range [0,1] and represents the probabiilty, that y=1.  The black dots show the training data.  We'll compute the the likelihood and the negative log likelihood."
+        "The left is model output and the right is the model output after the sigmoid has been applied, so it now lies in the range [0,1] and represents the probability, that y=1.  The black dots show the training data.  We'll compute the likelihood and the negative log likelihood."
       ],
       "metadata": {
         "id": "MvVX6tl9AEXF"
@@ -210,7 +210,7 @@
       "source": [
         "# Return probability under Bernoulli distribution for input x\n",
         "def bernoulli_distribution(y, lambda_param):\n",
-        "    # TODO-- write in the equation for the Bernoullid distribution \n",
+        "    # TODO-- write in the equation for the Bernoulli distribution \n",
         "    # Equation 5.17 from the notes (you will need np.power)\n",
         "    # Replace the line below\n",
         "    prob = np.zeros_like(y)\n",
@@ -249,7 +249,7 @@
       "source": [
         "# Return the likelihood of all of the data under the model\n",
         "def compute_likelihood(y_train, lambda_param):\n",
-        "  # TODO -- compute the likelihood of the data -- the product of the Bernoullis probabilities for each data point\n",
+        "  # TODO -- compute the likelihood of the data -- the product of the Bernoulli's probabilities for each data point\n",
         "  # Top line of equation 5.3 in the notes\n",
         "  # You will need np.prod() and the bernoulli_distribution function you used above\n",
         "  # Replace the line below\n",
@@ -284,7 +284,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "You can see that this gives a very small answer, even for this small 1D dataset, and with the model fitting quite well.  This is because it is the product of sveral probabilities, which are all quite small themselves.\n",
+        "You can see that this gives a very small answer, even for this small 1D dataset, and with the model fitting quite well.  This is because it is the product of several probabilities, which are all quite small themselves.\n",
         "This will get out of hand pretty quickly with real datasets -- the likelihood will get so small that we can't represent it with normal finite-precision math\n",
         "\n",
         "This is why we use negative log likelihood"
@@ -317,7 +317,7 @@
         "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
         "# Use our neural network to predict the mean of the Gaussian\n",
         "model_out = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
-        "# Set the standard devation to something reasonable\n",
+        "# Set the standard deviation to something reasonable\n",
         "lambda_train = sigmoid(model_out)\n",
         "# Compute the log likelihood\n",
         "nll = compute_negative_log_likelihood(y_train, lambda_train)\n",
@@ -362,7 +362,7 @@
       "source": [
         "# Define a range of values for the parameter\n",
         "beta_1_vals = np.arange(-2,6.0,0.1)\n",
-        "# Create some arrays to store the likelihoods, negative log likehoods\n",
+        "# Create some arrays to store the likelihoods, negative log likelihoods\n",
         "likelihoods = np.zeros_like(beta_1_vals)\n",
         "nlls = np.zeros_like(beta_1_vals)\n",
         "\n",

CM20315/CM20315_Loss_III.ipynb

@@ -33,7 +33,7 @@
         "# Loss functions part III\n",
         "\n",
         "This practical investigates loss functions.  In part I we investigated univariate regression (where the output data $y$ is continuous.  Our formulation was based on the normal/Gaussian distribution.\n",
-        "In part II we investigated binary classification (where the output data is 0 or 1).  This will be based on the Bernouilli distribution.<br><br>\n",
+        "In part II we investigated binary classification (where the output data is 0 or 1).  This will be based on the Bernoulli distribution.<br><br>\n",
         "\n",
         "Now we'll investigate multiclass classification (where the outputs data can take multiple values 1,... K, which is based on the categorical distribution\n",
         "\n",
@@ -218,7 +218,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "The left is model output and the right is the model output after the softmax has been applied, so it now lies in the range [0,1] and represents the probabiilty, that y=0 (red), 1 (green) and 2 (blue)   The dots at the bottom show the training data with the same color scheme.  So we want the red curve to be high where there are red dots, the green curve to be high where there are green dotsmand the blue curve to be high where there are blue dots  We'll compute the the likelihood and the negative log likelihood."
+        "The left is model output and the right is the model output after the softmax has been applied, so it now lies in the range [0,1] and represents the probability, that y=0 (red), 1 (green) and 2 (blue)   The dots at the bottom show the training data with the same color scheme.  So we want the red curve to be high where there are red dots, the green curve to be high where there are green dotsmand the blue curve to be high where there are blue dots  We'll compute the likelihood and the negative log likelihood."
       ],
       "metadata": {
         "id": "MvVX6tl9AEXF"
@@ -228,7 +228,7 @@
       "cell_type": "code",
       "source": [
         "# Return probability under Bernoulli distribution for input x\n",
-        "# Complicated code to commpute it but just take value from row k of lambda param where y =k, \n",
+        "# Complicated code to compute it but just take value from row k of lambda param where y =k, \n",
         "def categorical_distribution(y, lambda_param):\n",
         "    prob = np.zeros_like(y)\n",
         "    for row_index in range(lambda_param.shape[0]):\n",
@@ -305,7 +305,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "You can see that this gives a very small answer, even for this small 1D dataset, and with the model fitting quite well.  This is because it is the product of sveral probabilities, which are all quite small themselves.\n",
+        "You can see that this gives a very small answer, even for this small 1D dataset, and with the model fitting quite well.  This is because it is the product of several probabilities, which are all quite small themselves.\n",
         "This will get out of hand pretty quickly with real datasets -- the likelihood will get so small that we can't represent it with normal finite-precision math\n",
         "\n",
         "This is why we use negative log likelihood"
@@ -338,7 +338,7 @@
         "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
         "# Use our neural network to predict the mean of the Gaussian\n",
         "model_out = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
-        "# Set the standard devation to something reasonable\n",
+        "# Set the standard deviation to something reasonable\n",
         "lambda_train = softmax(model_out)\n",
         "# Compute the log likelihood\n",
         "nll = compute_negative_log_likelihood(y_train, lambda_train)\n",
@@ -365,7 +365,7 @@
       "source": [
         "# Define a range of values for the parameter\n",
         "beta_1_vals = np.arange(-2,6.0,0.1)\n",
-        "# Create some arrays to store the likelihoods, negative log likehoods\n",
+        "# Create some arrays to store the likelihoods, negative log likelihoods\n",
         "likelihoods = np.zeros_like(beta_1_vals)\n",
         "nlls = np.zeros_like(beta_1_vals)\n",
         "\n",

CM20315/CM20315_Shallow.ipynb

@@ -233,7 +233,7 @@
         "# TODO\n",
         "# 1. Predict what effect changing phi_0 will have on the network.  \n",
         "#   Answer:\n",
-        "# 2. Predict what effect multplying phi_1, phi_2, phi_3 by 0.5 would have.  Check if you are correct\n",
+        "# 2. Predict what effect multiplying phi_1, phi_2, phi_3 by 0.5 would have.  Check if you are correct\n",
         "#   Answer:\n",
         "# 3. Predict what effect multiplying phi_1 by -1 will have.  Check if you are correct.\n",
         "#   Answer:\n",
@@ -500,7 +500,7 @@
         "print(\"Loss = %3.3f\"%(loss))\n",
         "\n",
         "# TODO.  Manipulate the parameters (by hand!) to make the function \n",
-        "# fit the data better and try to reduct the loss to as small a number \n",
+        "# fit the data better and try to reduce the loss to as small a number \n",
         "# as possible.  The best that I could do was 0.181\n",
         "# Tip... start by manipulating phi_0.\n",
         "# It's not that easy, so don't spend too much time on this!"

CM20315/CM20315_Training_I.ipynb

@@ -108,7 +108,7 @@
       "source": [
         "def line_search(loss_function, thresh=.0001, max_iter = 10, draw_flag = False):\n",
         "\n",
-        "    # Initialize four points along the rnage we are going to search\n",
+        "    # Initialize four points along the range we are going to search\n",
         "    a = 0\n",
         "    b = 0.33\n",
         "    c = 0.66\n",
@@ -139,7 +139,7 @@
         "        # Rule #2 If point b is less than point c then\n",
         "        #                     then point d becomes point c, and\n",
         "        #                     point b becomes 1/3 between a and new d\n",
-        "        #                     point c beocome 2/3 between a and new d \n",
+        "        #                     point c becomes 2/3 between a and new d \n",
         "        # TODO REPLACE THE BLOCK OF CODE BELOW WITH THIS RULE\n",
         "        if (0):\n",
         "          continue;\n",
@@ -147,7 +147,7 @@
         "        # Rule #3 If point c is less than point b then\n",
         "        #                     then point a becomes point b, and\n",
         "        #                     point b becomes 1/3 between new a and d\n",
-        "        #                     point c beocome 2/3 between new a and d \n",
+        "        #                     point c becomes 2/3 between new a and d \n",
         "        # TODO REPLACE THE BLOCK OF CODE BELOW WITH THIS RULE\n",
         "        if(0):\n",
         "          continue\n",

CM20315/CM20315_Training_II.ipynb

@@ -114,7 +114,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Initialize the parmaeters and draw the model\n",
+        "# Initialize the parameters and draw the model\n",
         "phi = np.zeros((2,1))\n",
         "phi[0] = 0.6      # Intercept\n",
         "phi[1] = -0.2      # Slope\n",
@@ -314,7 +314,7 @@
         "  return compute_loss(data[0,:], data[1,:], model, phi_start+ gradient * dist_prop)\n",
         "\n",
         "def line_search(data, model, phi, gradient, thresh=.00001, max_dist = 0.1, max_iter = 15, verbose=False):\n",
-        "    # Initialize four points along the rnage we are going to search\n",
+        "    # Initialize four points along the range we are going to search\n",
         "    a = 0\n",
         "    b = 0.33 * max_dist\n",
         "    c = 0.66 * max_dist\n",
@@ -345,7 +345,7 @@
         "        # Rule #2 If point b is less than point c then\n",
         "        #                     then point d becomes point c, and\n",
         "        #                     point b becomes 1/3 between a and new d\n",
-        "        #                     point c beocome 2/3 between a and new d \n",
+        "        #                     point c becomes 2/3 between a and new d \n",
         "        if lossb < lossc:\n",
         "          d = c\n",
         "          b = a+ (d-a)/3\n",
@@ -355,7 +355,7 @@
         "        # Rule #2 If point c is less than point b then\n",
         "        #                     then point a becomes point b, and\n",
         "        #                     point b becomes 1/3 between new a and d\n",
-        "        #                     point c beocome 2/3 between new a and d \n",
+        "        #                     point c becomes 2/3 between new a and d \n",
         "        a = b\n",
         "        b = a+ (d-a)/3\n",
         "        c = a+ 2*(d-a)/3\n",

CM20315/CM20315_Training_III.ipynb

@@ -340,7 +340,7 @@
         "  return compute_loss(data[0,:], data[1,:], model, phi_start+ gradient * dist_prop)\n",
         "\n",
         "def line_search(data, model, phi, gradient, thresh=.00001, max_dist = 0.1, max_iter = 15, verbose=False):\n",
-        "    # Initialize four points along the rnage we are going to search\n",
+        "    # Initialize four points along the range we are going to search\n",
         "    a = 0\n",
         "    b = 0.33 * max_dist\n",
         "    c = 0.66 * max_dist\n",
@@ -371,7 +371,7 @@
         "        # Rule #2 If point b is less than point c then\n",
         "        #                     then point d becomes point c, and\n",
         "        #                     point b becomes 1/3 between a and new d\n",
-        "        #                     point c beocome 2/3 between a and new d \n",
+        "        #                     point c becomes 2/3 between a and new d \n",
         "        if lossb < lossc:\n",
         "          d = c\n",
         "          b = a+ (d-a)/3\n",
@@ -381,7 +381,7 @@
         "        # Rule #2 If point c is less than point b then\n",
         "        #                     then point a becomes point b, and\n",
         "        #                     point b becomes 1/3 between new a and d\n",
-        "        #                     point c beocome 2/3 between new a and d \n",
+        "        #                     point c becomes 2/3 between new a and d \n",
         "        a = b\n",
         "        b = a+ (d-a)/3\n",
         "        c = a+ 2*(d-a)/3\n",

CM20315/CM20315_Transformers.ipynb

@@ -175,7 +175,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# TODO Modify the code below by changeing the number of tokens generated and the initial sentence\n",
+        "# TODO Modify the code below by changing the number of tokens generated and the initial sentence\n",
         "# to get a feel for how well this works.  Since I didn't reset the seed, it will give a different\n",
         "# answer every time that you run it.\n",
         "\n",
@@ -253,7 +253,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# TODO Modify the code below by changeing the number of tokens generated and the initial sentence\n",
+        "# TODO Modify the code below by changing the number of tokens generated and the initial sentence\n",
         "# to get a feel for how well this works.  \n",
         "\n",
         "# TODO Experiment with changing this line:\n",
@@ -471,7 +471,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# This routine reutnrs the k'th most likely next token.\n",
+        "# This routine returns the k'th most likely next token.\n",
         "# If k =0 then it returns the most likely token, if k=1 it returns the next most likely and so on\n",
         "# We will need this for beam search\n",
         "def get_kth_most_likely_token(input_tokens, model, tokenizer, k):\n",

CM20315_2023/CM20315_Coursework_I.ipynb

@@ -0,0 +1,280 @@
+{
+  "nbformat": 4,
+  "nbformat_minor": 0,
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "authorship_tag": "ABX9TyPNASgWoh4kBvxFP0xkN/I4",
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "name": "python3",
+      "display_name": "Python 3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "view-in-github",
+        "colab_type": "text"
+      },
+      "source": [
+        "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/CM20315_2023/CM20315_Coursework_I.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "# Coursework I -- Model hyperparameters\n",
+        "\n",
+        "The goal of the coursework is to modify a simple bit of numpy code that trains a network and measures the performance on a validation set for the MNIST 1D dataset.\n",
+        "\n",
+        "In this coursework, you need to modify the **model hyperparameters** (only) to improve the performance over the current attempt.  This could mean the number of layers, the number of hidden units per layer, or the type of activation function, or any combination of the three.\n",
+        "\n",
+        "The only constraint is that you MUST use a fully connected network (no convolutional networks for now if you have read ahead in the book).\n",
+        "\n",
+        "You must improve the performance by at least 2% to get full marks.\n",
+        "\n",
+        "You will need to upload three things to Moodle:\n",
+        "1.   The image that this notebook saves (click the folder icon on the left on colab to download it)\n",
+        "2.   The lines of code you changed\n",
+        "3.   The whole notebook as a .ipynb file.  You can do this on the File menu\n",
+        "\n",
+        "\n"
+      ],
+      "metadata": {
+        "id": "t9vk9Elugvmi"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "import numpy as np\n",
+        "import os\n",
+        "import torch, torch.nn as nn\n",
+        "from torch.utils.data import TensorDataset, DataLoader\n",
+        "from torch.optim.lr_scheduler import StepLR\n",
+        "import matplotlib.pyplot as plt\n",
+        "import random\n",
+        "import gdown"
+      ],
+      "metadata": {
+        "id": "YrXWAH7sUWvU"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "\n",
+        "# Run this once to copy the train and validation data to your CoLab environment\n",
+        "# or download from my github to your local machine if you are doing this locally\n",
+        "if not os.path.exists('./Data.zip'):\n",
+        "  !gdown 1HtnCrncY6dFCYqzgPf1HtPVAerTpwFRm\n",
+        "  !unzip Data.zip"
+      ],
+      "metadata": {
+        "id": "wScBGXXFVadm"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Load in the data\n",
+        "train_data_x = np.load('train_data_x.npy')\n",
+        "val_data_y = np.load('val_data_y.npy')\n",
+        "train_data_y = np.load('train_data_y.npy')\n",
+        "val_data_x = np.load('val_data_x.npy')\n",
+        "# Print out sizes\n",
+        "print(\"Train data: %d examples (columns), each of which has %d dimensions (rows)\"%((train_data_x.shape[1],train_data_x.shape[0])))\n",
+        "print(\"Validation data: %d examples (columns), each of which has %d dimensions (rows)\"%((val_data_x.shape[1],val_data_x.shape[0])))"
+      ],
+      "metadata": {
+        "id": "8bKADvLHbiV5"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "Define the network"
+      ],
+      "metadata": {
+        "id": "_sFvRDGrl4qe"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# YOU SHOULD ONLY CHANGE THIS CELL!\n",
+        "\n",
+        "# There are 40 input dimensions and 10 output dimensions for this data\n",
+        "# The inputs correspond to the 40 offsets in the MNIST1D template.\n",
+        "D_i = 40\n",
+        "# The outputs correspond to the 10 digits\n",
+        "D_o = 10\n",
+        "\n",
+        "# Number of hidden units in layers 1 and 2\n",
+        "D_1 = 100\n",
+        "D_2 = 100\n",
+        "\n",
+        "# create model with two hidden layers\n",
+        "model = nn.Sequential(\n",
+        "nn.Linear(D_i, D_1),\n",
+        "nn.ReLU(),\n",
+        "nn.Linear(D_1, D_2),\n",
+        "nn.ReLU(),\n",
+        "nn.Linear(D_2, D_o))"
+      ],
+      "metadata": {
+        "id": "FslroPJJffrh"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# He initialization of weights\n",
+        "def weights_init(layer_in):\n",
+        "  if isinstance(layer_in, nn.Linear):\n",
+        "    nn.init.kaiming_uniform_(layer_in.weight)\n",
+        "    layer_in.bias.data.fill_(0.0)"
+      ],
+      "metadata": {
+        "id": "YgLaex1pfhqz"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# You need all this stuff to ensure that PyTorch is deterministic\n",
+        "def set_seed(seed):\n",
+        "    torch.manual_seed(seed)\n",
+        "    torch.cuda.manual_seed_all(seed)\n",
+        "    torch.backends.cudnn.deterministic = True\n",
+        "    torch.backends.cudnn.benchmark = False\n",
+        "    np.random.seed(seed)\n",
+        "    random.seed(seed)\n",
+        "    os.environ['PYTHONHASHSEED'] = str(seed)"
+      ],
+      "metadata": {
+        "id": "zXRmxCQNnL_M"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Set seed so always get same result (do not change)\n",
+        "set_seed(1)\n",
+        "\n",
+        "# choose cross entropy loss function (equation 5.24 in the loss notes)\n",
+        "loss_function = nn.CrossEntropyLoss()\n",
+        "# construct SGD optimizer and initialize learning rate and momentum\n",
+        "optimizer = torch.optim.SGD(model.parameters(), lr = 0.05, momentum=0.9)\n",
+        "# object that decreases learning rate by half every 10 epochs\n",
+        "scheduler = StepLR(optimizer, step_size=10, gamma=0.5)\n",
+        "# create 100 dummy data points and store in data loader class\n",
+        "x_train = torch.tensor(train_data_x.transpose().astype('float32'))\n",
+        "y_train = torch.tensor(train_data_y.astype('long'))\n",
+        "x_val= torch.tensor(val_data_x.transpose().astype('float32'))\n",
+        "y_val = torch.tensor(val_data_y.astype('long'))\n",
+        "\n",
+        "# load the data into a class that creates the batches\n",
+        "data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
+        "\n",
+        "# Initialize model weights\n",
+        "model.apply(weights_init)\n",
+        "\n",
+        "# loop over the dataset n_epoch times\n",
+        "n_epoch = 50\n",
+        "# store the loss and the % correct at each epoch\n",
+        "losses_train = np.zeros((n_epoch))\n",
+        "errors_train = np.zeros((n_epoch))\n",
+        "losses_val = np.zeros((n_epoch))\n",
+        "errors_val = np.zeros((n_epoch))\n",
+        "\n",
+        "for epoch in range(n_epoch):\n",
+        "  # loop over batches\n",
+        "  for i, data in enumerate(data_loader):\n",
+        "    # retrieve inputs and labels for this batch\n",
+        "    x_batch, y_batch = data\n",
+        "    # zero the parameter gradients\n",
+        "    optimizer.zero_grad()\n",
+        "    # forward pass -- calculate model output\n",
+        "    pred = model(x_batch)\n",
+        "    # compute the lss\n",
+        "    loss = loss_function(pred, y_batch)\n",
+        "    # backward pass\n",
+        "    loss.backward()\n",
+        "    # SGD update\n",
+        "    optimizer.step()\n",
+        "\n",
+        "  # Run whole dataset to get statistics -- normally wouldn't do this\n",
+        "  pred_train = model(x_train)\n",
+        "  pred_val = model(x_val)\n",
+        "  _, predicted_train_class = torch.max(pred_train.data, 1)\n",
+        "  _, predicted_val_class = torch.max(pred_val.data, 1)\n",
+        "  errors_train[epoch] = 100 - 100 * (predicted_train_class == y_train).float().sum() / len(y_train)\n",
+        "  errors_val[epoch]= 100 - 100 * (predicted_val_class == y_val).float().sum() / len(y_val)\n",
+        "  losses_train[epoch] = loss_function(pred_train, y_train).item()\n",
+        "  losses_val[epoch]= loss_function(pred_val, y_val).item()\n",
+        "  print(f'Epoch {epoch:5d}, train loss {losses_train[epoch]:.6f}, train error {errors_train[epoch]:3.2f},  val loss {losses_val[epoch]:.6f}, percent error {errors_val[epoch]:3.2f}')\n",
+        "\n",
+        "  # tell scheduler to consider updating learning rate\n",
+        "  scheduler.step()\n",
+        "\n",
+        "# Plot the results\n",
+        "fig, ax = plt.subplots()\n",
+        "ax.plot(errors_train,'r-',label='train')\n",
+        "ax.plot(errors_val,'b-',label='validation')\n",
+        "ax.set_ylim(0,100); ax.set_xlim(0,n_epoch)\n",
+        "ax.set_xlabel('Epoch'); ax.set_ylabel('Error')\n",
+        "ax.set_title('Part I: Validation Result %3.2f'%(errors_val[-1]))\n",
+        "ax.legend()\n",
+        "ax.plot([0,n_epoch],[37.45, 37.45],'k:') # Original results. You should be better than this!\n",
+        "plt.savefig('Coursework_I_Results.png',format='png')\n",
+        "plt.show()"
+      ],
+      "metadata": {
+        "id": "NYw8I_3mmX5c"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Leave this all commented for now\n",
+        "# We'll see how well you did on the test data after the coursework is submitted\n",
+        "\n",
+        "# # I haven't given you this yet, leave commented\n",
+        "# test_data_x = np.load('test_data_x.npy')\n",
+        "# test_data_y = np.load('test_data_y.npy')\n",
+        "# x_test = torch.tensor(test_data_x.transpose().astype('float32'))\n",
+        "# y_test = torch.tensor(test_data_y.astype('long'))\n",
+        "# pred_test = model(x_test)\n",
+        "# _, predicted_test_class = torch.max(pred_test.data, 1)\n",
+        "# errors_test = 100 - 100 * (predicted_test_class == y_test).float().sum() / len(y_test)\n",
+        "# print(\"Test error = %3.3f\"%(errors_test))"
+      ],
+      "metadata": {
+        "id": "O7nBz-R84QdJ"
+      },
+      "execution_count": null,
+      "outputs": []
+    }
+  ]
+}

CM20315_2023/CM20315_Coursework_II.ipynb

@@ -0,0 +1,276 @@
+{
+  "nbformat": 4,
+  "nbformat_minor": 0,
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "authorship_tag": "ABX9TyM+iKos5DEoHUxL8+9oxA2A",
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "name": "python3",
+      "display_name": "Python 3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "view-in-github",
+        "colab_type": "text"
+      },
+      "source": [
+        "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/CM20315_2023/CM20315_Coursework_II.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "# Coursework II -- Training hyperparameters\n",
+        "\n",
+        "The goal of the coursework is to modify a simple bit of numpy code that trains a network and measures the performance on a validation set for the MNist 1D dataset.\n",
+        "\n",
+        "In this coursework, you need to modify the **training hyperparameters** (only) to improve the performance over the current attempt. This could mean the training algorithm, learning rate, learning rate schedule, momentum term, initialization etc.  \n",
+        "\n",
+        "You must improve the performance by at least 2% to get full marks.\n",
+        "\n",
+        "You will need to upload three things to Moodle:\n",
+        "1.   The image that this notebook saves (click the folder icon on the left on colab to download it)\n",
+        "2.   The lines of code you changed\n",
+        "3.   The whole notebook as a .ipynb file.  You can do this on the File menu"
+      ],
+      "metadata": {
+        "id": "t9vk9Elugvmi"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "import numpy as np\n",
+        "import os\n",
+        "import torch, torch.nn as nn\n",
+        "from torch.utils.data import TensorDataset, DataLoader\n",
+        "from torch.optim.lr_scheduler import StepLR\n",
+        "import matplotlib.pyplot as plt\n",
+        "import random\n",
+        "import gdown"
+      ],
+      "metadata": {
+        "id": "YrXWAH7sUWvU"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Run this once to copy the train and validation data to your CoLab environment\n",
+        "if not os.path.exists('./Data.zip'):\n",
+        "  !gdown 1HtnCrncY6dFCYqzgPf1HtPVAerTpwFRm\n",
+        "  !unzip Data.zip"
+      ],
+      "metadata": {
+        "id": "wScBGXXFVadm"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Load in the data\n",
+        "train_data_x = np.load('train_data_x.npy',allow_pickle=True)\n",
+        "train_data_y = np.load('train_data_y.npy',allow_pickle=True)\n",
+        "val_data_x = np.load('val_data_x.npy',allow_pickle=True)\n",
+        "val_data_y = np.load('val_data_y.npy',allow_pickle=True)\n",
+        "# Print out sizes\n",
+        "print(\"Train data: %d examples (columns), each of which has %d dimensions (rows)\"%((train_data_x.shape[1],train_data_x.shape[0])))\n",
+        "print(\"Validation data: %d examples (columns), each of which has %d dimensions (rows)\"%((val_data_x.shape[1],val_data_x.shape[0])))"
+      ],
+      "metadata": {
+        "id": "8bKADvLHbiV5"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "Define the network"
+      ],
+      "metadata": {
+        "id": "_sFvRDGrl4qe"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# YOU SHOULD NOT CHANGE THIS CELL!\n",
+        "\n",
+        "# There are 40 input dimensions and 10 output dimensions for this data\n",
+        "# The inputs correspond to the 40 offsets in the MNIST1D template.\n",
+        "D_i = 40\n",
+        "# The outputs correspond to the 10 digits\n",
+        "D_o = 10\n",
+        "\n",
+        "# Number of hidden units in layers 1 and 2\n",
+        "D_1 = 100\n",
+        "D_2 = 100\n",
+        "\n",
+        "# create model with two hidden layers\n",
+        "model = nn.Sequential(\n",
+        "nn.Linear(D_i, D_1),\n",
+        "nn.ReLU(),\n",
+        "nn.Linear(D_1, D_2),\n",
+        "nn.ReLU(),\n",
+        "nn.Linear(D_2, D_o))"
+      ],
+      "metadata": {
+        "id": "FslroPJJffrh"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# He initialization of weights\n",
+        "def weights_init(layer_in):\n",
+        "  if isinstance(layer_in, nn.Linear):\n",
+        "    nn.init.kaiming_uniform_(layer_in.weight)\n",
+        "    layer_in.bias.data.fill_(0.0)"
+      ],
+      "metadata": {
+        "id": "YgLaex1pfhqz"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# You need all this stuff to ensure that PyTorch is deterministic\n",
+        "def set_seed(seed):\n",
+        "    torch.manual_seed(seed)\n",
+        "    torch.cuda.manual_seed_all(seed)\n",
+        "    torch.backends.cudnn.deterministic = True\n",
+        "    torch.backends.cudnn.benchmark = False\n",
+        "    np.random.seed(seed)\n",
+        "    random.seed(seed)\n",
+        "    os.environ['PYTHONHASHSEED'] = str(seed)"
+      ],
+      "metadata": {
+        "id": "zXRmxCQNnL_M"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Set seed so always get same result (do not change)\n",
+        "set_seed(1)\n",
+        "\n",
+        "# choose cross entropy loss function (equation 5.24 in the loss notes)\n",
+        "loss_function = nn.CrossEntropyLoss()\n",
+        "# construct SGD optimizer and initialize learning rate and momentum\n",
+        "optimizer = torch.optim.SGD(model.parameters(), lr = 0.05, momentum=0.9)\n",
+        "# object that decreases learning rate by half every 10 epochs\n",
+        "scheduler = StepLR(optimizer, step_size=10, gamma=0.5)\n",
+        "# create 100 dummy data points and store in data loader class\n",
+        "x_train = torch.tensor(train_data_x.transpose().astype('float32'))\n",
+        "print(x_train.shape)\n",
+        "y_train = torch.tensor(train_data_y.astype('long'))\n",
+        "print(y_train.shape)\n",
+        "x_val= torch.tensor(val_data_x.transpose().astype('float32'))\n",
+        "y_val = torch.tensor(val_data_y.astype('long'))\n",
+        "\n",
+        "# load the data into a class that creates the batches\n",
+        "data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
+        "\n",
+        "# Initialize model weights\n",
+        "model.apply(weights_init)\n",
+        "\n",
+        "# loop over the dataset n_epoch times\n",
+        "n_epoch = 50\n",
+        "# store the loss and the % correct at each epoch\n",
+        "losses_train = np.zeros((n_epoch))\n",
+        "errors_train = np.zeros((n_epoch))\n",
+        "losses_val = np.zeros((n_epoch))\n",
+        "errors_val = np.zeros((n_epoch))\n",
+        "\n",
+        "for epoch in range(n_epoch):\n",
+        "  # loop over batches\n",
+        "  for i, data in enumerate(data_loader):\n",
+        "    # retrieve inputs and labels for this batch\n",
+        "    x_batch, y_batch = data\n",
+        "    # zero the parameter gradients\n",
+        "    optimizer.zero_grad()\n",
+        "    # forward pass -- calculate model output\n",
+        "    pred = model(x_batch)\n",
+        "    # compute the lss\n",
+        "    loss = loss_function(pred, y_batch)\n",
+        "    # backward pass\n",
+        "    loss.backward()\n",
+        "    # SGD update\n",
+        "    optimizer.step()\n",
+        "\n",
+        "  # Run whole dataset to get statistics -- normally wouldn't do this\n",
+        "  pred_train = model(x_train)\n",
+        "  pred_val = model(x_val)\n",
+        "  _, predicted_train_class = torch.max(pred_train.data, 1)\n",
+        "  _, predicted_val_class = torch.max(pred_val.data, 1)\n",
+        "  errors_train[epoch] = 100 - 100 * (predicted_train_class == y_train).float().sum() / len(y_train)\n",
+        "  errors_val[epoch]= 100 - 100 * (predicted_val_class == y_val).float().sum() / len(y_val)\n",
+        "  losses_train[epoch] = loss_function(pred_train, y_train).item()\n",
+        "  losses_val[epoch]= loss_function(pred_val, y_val).item()\n",
+        "  print(f'Epoch {epoch:5d}, train loss {losses_train[epoch]:.6f}, train error {errors_train[epoch]:3.2f},  val loss {losses_val[epoch]:.6f}, percent error {errors_val[epoch]:3.2f}')\n",
+        "\n",
+        "  # tell scheduler to consider updating learning rate\n",
+        "  scheduler.step()\n",
+        "\n",
+        "# Plot the results\n",
+        "fig, ax = plt.subplots()\n",
+        "ax.plot(errors_train,'r-',label='train')\n",
+        "ax.plot(errors_val,'b-',label='validation')\n",
+        "ax.set_ylim(0,100); ax.set_xlim(0,n_epoch)\n",
+        "ax.set_xlabel('Epoch'); ax.set_ylabel('Error')\n",
+        "ax.set_title('Part II: Validation Result %3.2f'%(errors_val[-1]))\n",
+        "ax.legend()\n",
+        "ax.plot([0,n_epoch],[37.45, 37.45],'k:') # Original results. You should be better than this!\n",
+        "plt.savefig('Coursework_II_Results.png',format='png')\n",
+        "plt.show()"
+      ],
+      "metadata": {
+        "id": "NYw8I_3mmX5c"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Leave this all commented for now\n",
+        "# We'll see how well you did on the test data after the coursework is submitted\n",
+        "\n",
+        "# # I haven't given you this yet, leave commented\n",
+        "# test_data_x = np.load('test_data_x.npy')\n",
+        "# test_data_y = np.load('test_data_y.npy')\n",
+        "# x_test = torch.tensor(test_data_x.transpose().astype('float32'))\n",
+        "# y_test = torch.tensor(test_data_y.astype('long'))\n",
+        "# pred_test = model(x_test)\n",
+        "# _, predicted_test_class = torch.max(pred_test.data, 1)\n",
+        "# errors_test = 100 - 100 * (predicted_test_class == y_test).float().sum() / len(y_test)\n",
+        "# print(\"Test error = %3.3f\"%(errors_test))"
+      ],
+      "metadata": {
+        "id": "O7nBz-R84QdJ"
+      },
+      "execution_count": null,
+      "outputs": []
+    }
+  ]
+}

CM20315_2023/CM20315_Coursework_III.ipynb

@@ -0,0 +1,275 @@
+{
+  "nbformat": 4,
+  "nbformat_minor": 0,
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "authorship_tag": "ABX9TyNDH1z3I76jjglu3o0LSlZc",
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "name": "python3",
+      "display_name": "Python 3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "view-in-github",
+        "colab_type": "text"
+      },
+      "source": [
+        "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/CM20315_2023/CM20315_Coursework_III.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "# Coursework III -- Regularization\n",
+        "\n",
+        "The goal of the coursework is to modify a simple bit of numpy code that trains a network and measures the performance on a validation set for the MNist 1D dataset.\n",
+        "\n",
+        "In this coursework, you need add **regularization** of some kind to improve the performance.  Anything from chapter 9 of the book or anything else you can find is fine *except* early stopping.  You must not change the model hyperparameters or the training algorithm.\n",
+        "\n",
+        "You must improve the performance by at least 2% to get full marks.\n",
+        "\n",
+        "You will need to upload three things to Moodle:\n",
+        "1.   The image that this notebook saves (click the folder icon on the left on colab to download it)\n",
+        "2.   The lines of code you changed\n",
+        "3.   The whole notebook as a .ipynb file.  You can do this on the File menu\n",
+        "\n",
+        "\n"
+      ],
+      "metadata": {
+        "id": "t9vk9Elugvmi"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "import numpy as np\n",
+        "import os\n",
+        "import torch, torch.nn as nn\n",
+        "from torch.utils.data import TensorDataset, DataLoader\n",
+        "from torch.optim.lr_scheduler import StepLR\n",
+        "import matplotlib.pyplot as plt\n",
+        "import random\n",
+        "import gdown"
+      ],
+      "metadata": {
+        "id": "YrXWAH7sUWvU"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Run this once to copy the train and validation data to your CoLab environment\n",
+        "if not os.path.exists('./Data.zip'):\n",
+        "  !gdown 1HtnCrncY6dFCYqzgPf1HtPVAerTpwFRm\n",
+        "  !unzip Data.zip"
+      ],
+      "metadata": {
+        "id": "wScBGXXFVadm"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Load in the data\n",
+        "train_data_x = np.load('train_data_x.npy')\n",
+        "train_data_y = np.load('train_data_y.npy')\n",
+        "val_data_x = np.load('val_data_x.npy')\n",
+        "val_data_y = np.load('val_data_y.npy')\n",
+        "# Print out sizes\n",
+        "print(\"Train data: %d examples (columns), each of which has %d dimensions (rows)\"%((train_data_x.shape[1],train_data_x.shape[0])))\n",
+        "print(\"Validation data: %d examples (columns), each of which has %d dimensions (rows)\"%((val_data_x.shape[1],val_data_x.shape[0])))"
+      ],
+      "metadata": {
+        "id": "8bKADvLHbiV5"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "Define the network"
+      ],
+      "metadata": {
+        "id": "_sFvRDGrl4qe"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# There are 40 input dimensions and 10 output dimensions for this data\n",
+        "# The inputs correspond to the 40 offsets in the MNIST1D template.\n",
+        "D_i = 40\n",
+        "# The outputs correspond to the 10 digits\n",
+        "D_o = 10\n",
+        "\n",
+        "# Number of hidden units in layers 1 and 2\n",
+        "D_1 = 100\n",
+        "D_2 = 100\n",
+        "\n",
+        "# create model with two hidden layers\n",
+        "model = nn.Sequential(\n",
+        "nn.Linear(D_i, D_1),\n",
+        "nn.ReLU(),\n",
+        "nn.Linear(D_1, D_2),\n",
+        "nn.ReLU(),\n",
+        "nn.Linear(D_2, D_o))"
+      ],
+      "metadata": {
+        "id": "FslroPJJffrh"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# He initialization of weights\n",
+        "def weights_init(layer_in):\n",
+        "  if isinstance(layer_in, nn.Linear):\n",
+        "    nn.init.kaiming_uniform_(layer_in.weight)\n",
+        "    layer_in.bias.data.fill_(0.0)"
+      ],
+      "metadata": {
+        "id": "YgLaex1pfhqz"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# You need all this stuff to ensure that PyTorch is deterministic\n",
+        "def set_seed(seed):\n",
+        "    torch.manual_seed(seed)\n",
+        "    torch.cuda.manual_seed_all(seed)\n",
+        "    torch.backends.cudnn.deterministic = True\n",
+        "    torch.backends.cudnn.benchmark = False\n",
+        "    np.random.seed(seed)\n",
+        "    random.seed(seed)\n",
+        "    os.environ['PYTHONHASHSEED'] = str(seed)"
+      ],
+      "metadata": {
+        "id": "zXRmxCQNnL_M"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Set seed so always get same result (do not change)\n",
+        "set_seed(1)\n",
+        "\n",
+        "# choose cross entropy loss function (equation 5.24 in the loss notes)\n",
+        "loss_function = nn.CrossEntropyLoss()\n",
+        "# construct SGD optimizer and initialize learning rate and momentum\n",
+        "optimizer = torch.optim.SGD(model.parameters(), lr = 0.05, momentum=0.9)\n",
+        "# object that decreases learning rate by half every 10 epochs\n",
+        "scheduler = StepLR(optimizer, step_size=10, gamma=0.5)\n",
+        "# create 100 dummy data points and store in data loader class\n",
+        "x_train = torch.tensor(train_data_x.transpose().astype('float32'))\n",
+        "y_train = torch.tensor(train_data_y.astype('long'))\n",
+        "x_val= torch.tensor(val_data_x.transpose().astype('float32'))\n",
+        "y_val = torch.tensor(val_data_y.astype('long'))\n",
+        "\n",
+        "# load the data into a class that creates the batches\n",
+        "data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
+        "\n",
+        "# Initialize model weights\n",
+        "model.apply(weights_init)\n",
+        "\n",
+        "# loop over the dataset n_epoch times\n",
+        "n_epoch = 50\n",
+        "# store the loss and the % correct at each epoch\n",
+        "losses_train = np.zeros((n_epoch))\n",
+        "errors_train = np.zeros((n_epoch))\n",
+        "losses_val = np.zeros((n_epoch))\n",
+        "errors_val = np.zeros((n_epoch))\n",
+        "\n",
+        "for epoch in range(n_epoch):\n",
+        "  # loop over batches\n",
+        "  for i, data in enumerate(data_loader):\n",
+        "    # retrieve inputs and labels for this batch\n",
+        "    x_batch, y_batch = data\n",
+        "    # zero the parameter gradients\n",
+        "    optimizer.zero_grad()\n",
+        "    # forward pass -- calculate model output\n",
+        "    pred = model(x_batch)\n",
+        "    # compute the lss\n",
+        "    loss = loss_function(pred, y_batch)\n",
+        "    # backward pass\n",
+        "    loss.backward()\n",
+        "    # SGD update\n",
+        "    optimizer.step()\n",
+        "\n",
+        "  # Run whole dataset to get statistics -- normally wouldn't do this\n",
+        "  pred_train = model(x_train)\n",
+        "  pred_val = model(x_val)\n",
+        "  _, predicted_train_class = torch.max(pred_train.data, 1)\n",
+        "  _, predicted_val_class = torch.max(pred_val.data, 1)\n",
+        "  errors_train[epoch] = 100 - 100 * (predicted_train_class == y_train).float().sum() / len(y_train)\n",
+        "  errors_val[epoch]= 100 - 100 * (predicted_val_class == y_val).float().sum() / len(y_val)\n",
+        "  losses_train[epoch] = loss_function(pred_train, y_train).item()\n",
+        "  losses_val[epoch]= loss_function(pred_val, y_val).item()\n",
+        "  print(f'Epoch {epoch:5d}, train loss {losses_train[epoch]:.6f}, train error {errors_train[epoch]:3.2f},  val loss {losses_val[epoch]:.6f}, percent error {errors_val[epoch]:3.2f}')\n",
+        "\n",
+        "  # tell scheduler to consider updating learning rate\n",
+        "  scheduler.step()\n",
+        "\n",
+        "# Plot the results\n",
+        "fig, ax = plt.subplots()\n",
+        "ax.plot(errors_train,'r-',label='train')\n",
+        "ax.plot(errors_val,'b-',label='validation')\n",
+        "ax.set_ylim(0,100); ax.set_xlim(0,n_epoch)\n",
+        "ax.set_xlabel('Epoch'); ax.set_ylabel('Error')\n",
+        "ax.set_title('Part III: Validation Result %3.2f'%(errors_val[-1]))\n",
+        "ax.legend()\n",
+        "ax.plot([0,n_epoch],[37.45, 37.45],'k:') # Original results. You should be better than this!\n",
+        "plt.savefig('Coursework_III_Results.png',format='png')\n",
+        "plt.show()"
+      ],
+      "metadata": {
+        "id": "NYw8I_3mmX5c"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Leave this all commented for now\n",
+        "# We'll see how well you did on the test data after the coursework is submitted\n",
+        "\n",
+        "\n",
+        "# # I haven't given you this yet, leave commented\n",
+        "# test_data_x = np.load('test_data_x.npy')\n",
+        "# test_data_y = np.load('test_data_y.npy')\n",
+        "# x_test = torch.tensor(test_data_x.transpose().astype('float32'))\n",
+        "# y_test = torch.tensor(test_data_y.astype('long'))\n",
+        "# pred_test = model(x_test)\n",
+        "# _, predicted_test_class = torch.max(pred_test.data, 1)\n",
+        "# errors_test = 100 - 100 * (predicted_test_class == y_test).float().sum() / len(y_test)\n",
+        "# print(\"Test error = %3.3f\"%(errors_test))"
+      ],
+      "metadata": {
+        "id": "O7nBz-R84QdJ"
+      },
+      "execution_count": null,
+      "outputs": []
+    }
+  ]
+}

CM20315_2023/CM20315_Coursework_IV.ipynb

@@ -0,0 +1,212 @@
+{
+  "nbformat": 4,
+  "nbformat_minor": 0,
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "authorship_tag": "ABX9TyMrWYwQrwgJvDza1vhYK9WQ",
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "name": "python3",
+      "display_name": "Python 3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "view-in-github",
+        "colab_type": "text"
+      },
+      "source": [
+        "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/CM20315_2023/CM20315_Coursework_IV.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "# Coursework IV\n",
+        "\n",
+        "This coursework explores the geometry of high dimensional spaces.  It doesn't behave how you would expect and all your intuitions are wrong!  You will write code and it will give you three numerical answers that you need to type into Moodle."
+      ],
+      "metadata": {
+        "id": "EjLK-kA1KnYX"
+      }
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "4ESMmnkYEVAb"
+      },
+      "outputs": [],
+      "source": [
+        "import numpy as np\n",
+        "import matplotlib.pyplot as plt\n",
+        "import scipy.special as sci"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "# Part (a)\n",
+        "\n",
+        "In part (a) of the practical, we investigate how close random points are in 2D, 100D, and 1000D.   In each case, we generate 1000 points and calculate the Euclidean distance between each pair.  You should find that in 1000D, the furthest two points are only slightly further apart than the nearest points.  Weird!"
+      ],
+      "metadata": {
+        "id": "MonbPEitLNgN"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Fix the random seed so we all have the same random numbers\n",
+        "np.random.seed(0)\n",
+        "n_data = 1000\n",
+        "# Create 1000 data examples (columns) each with 2 dimensions (rows)\n",
+        "n_dim = 2\n",
+        "x_2D = np.random.normal(size=(n_dim,n_data))\n",
+        "# Create 1000 data examples (columns) each with 100 dimensions (rows)\n",
+        "n_dim = 100\n",
+        "x_100D = np.random.normal(size=(n_dim,n_data))\n",
+        "# Create 1000 data examples (columns) each with 1000 dimensions (rows)\n",
+        "n_dim = 1000\n",
+        "x_1000D = np.random.normal(size=(n_dim,n_data))\n",
+        "\n",
+        "# These values should be the same, otherwise your answer will be wrong\n",
+        "# Get in touch if they are not!\n",
+        "print('Sum of your data is %3.3f, Should be %3.3f'%(np.sum(x_1000D),1036.321))"
+      ],
+      "metadata": {
+        "id": "vZSHVmcWEk14"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "def distance_ratio(x):\n",
+        "  # TODO -- replace the two lines below to calculate the largest and smallest Euclidean distance between\n",
+        "  # the data points in the columns of x.  DO NOT include the distance between the data point\n",
+        "  # and itself (which is obviously zero)\n",
+        "  smallest_dist = 1.0\n",
+        "  largest_dist = 1.0\n",
+        "\n",
+        "  # Calculate the ratio and return\n",
+        "  dist_ratio = largest_dist / smallest_dist\n",
+        "  return dist_ratio"
+      ],
+      "metadata": {
+        "id": "PhVmnUs8ErD9"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "print('Ratio of largest to smallest distance 2D: %3.3f'%(distance_ratio(x_2D)))\n",
+        "print('Ratio of largest to smallest distance 100D: %3.3f'%(distance_ratio(x_100D)))\n",
+        "print('Ratio of largest to smallest distance 1000D: %3.3f'%(distance_ratio(x_1000D)))\n",
+        "print('**Note down the last of these three numbers, you will need to submit it for your coursework**')"
+      ],
+      "metadata": {
+        "id": "0NdPxfn5GQuJ"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "# Part (b)\n",
+        "\n",
+        "In part (b) of the practical we calculate the volume of a hypersphere of radius 0.5 (i.e., of diameter 1) as a function of the radius.  You will find that the volume decreases to almost nothing in high dimensions.  All of the volume is in the corners of the unit hypercube (which always has volume 1). Double weird.\n",
+        "\n",
+        "Note that you can check your answer by doing the calculation for 2D using the standard formula for the area of a circle and making sure it matches."
+      ],
+      "metadata": {
+        "id": "b2FYKV1SL4Z7"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "def volume_of_hypersphere(diameter, dimensions):\n",
+        "  # Formula given in Problem 8.7 of the notes\n",
+        "  # You will need sci.gamma()\n",
+        "  # Check out:    https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.gamma.html\n",
+        "  # Also use this value for pi\n",
+        "  pi = np.pi\n",
+        "  # TODO replace this code with formula for the volume of a hypersphere\n",
+        "  volume = 1.0\n",
+        "\n",
+        "  return volume\n"
+      ],
+      "metadata": {
+        "id": "CZoNhD8XJaHR"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "diameter = 1.0\n",
+        "for c_dim in range(1,11):\n",
+        "  print(\"Volume of unit diameter hypersphere in %d dimensions is %3.3f\"%(c_dim, volume_of_hypersphere(diameter, c_dim)))\n",
+        "print('**Note down the last of these ten numbers, you will need to submit it for your coursework**')"
+      ],
+      "metadata": {
+        "id": "fNTBlg_GPEUh"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "# Part (c)\n",
+        "\n",
+        "In part (c) of the coursework, you will calculate what proportion of the volume of a hypersphere is in the outer 1% of the radius/diameter.  Calculate the volume of a hypersphere and then the volume of a hypersphere with 0.99 of the radius and then figure out the proportion (a number between 0 and 1).  You'll see that by the time we get to 300 dimensions most of the volume is in the outer 1 percent.  Extremely weird!"
+      ],
+      "metadata": {
+        "id": "GdyMeOBmoXyF"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "def get_prop_of_volume_in_outer_1_percent(dimension):\n",
+        "  # TODO -- replace this line\n",
+        "  proportion = 1.0\n",
+        "\n",
+        "  return proportion"
+      ],
+      "metadata": {
+        "id": "8_CxZ2AIpQ8w"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# While we're here, let's look at how much of the volume is in the outer 1% of the radius\n",
+        "for c_dim in [1,2,10,20,50,100,150,200,250,300]:\n",
+        "  print('Proportion of volume in outer 1 percent of radius in %d dimensions =%3.3f'%(c_dim, get_prop_of_volume_in_outer_1_percent(c_dim)))\n",
+        "print('**Note down the last of these ten numbers, you will need to submit it for your coursework**')"
+      ],
+      "metadata": {
+        "id": "LtMDIn2qPVfJ"
+      },
+      "execution_count": null,
+      "outputs": []
+    }
+  ]
+}

CM20315_2023/CM20315_Coursework_V_2023.ipynb

@@ -0,0 +1,525 @@
+{
+  "nbformat": 4,
+  "nbformat_minor": 0,
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "authorship_tag": "ABX9TyN7KaQQ63bf52r+b5as0MkK",
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "name": "python3",
+      "display_name": "Python 3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "view-in-github",
+        "colab_type": "text"
+      },
+      "source": [
+        "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/CM20315_2023/CM20315_Coursework_V_2023.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "# **Coursework V: Backpropagation in Toy Model**\n",
+        "\n",
+        "This notebook computes the derivatives of a toy function similar (but different from) that in section 7.3 of the book.\n",
+        "\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.  At various points, you will get an answer that you need to copy into Moodle to be marked.\n",
+        "\n",
+        "Post to the content forum if you find any mistakes or need to clarify something."
+      ],
+      "metadata": {
+        "id": "pOZ6Djz0dhoy"
+      }
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "# Problem setting\n",
+        "\n",
+        "We're going to investigate how to take the derivatives of functions where one operation is composed with another, which is composed with a third and so on.  For example, consider the model:\n",
+        "\n",
+        "\\begin{equation}\n",
+        "     \\mbox{f}[x,\\boldsymbol\\phi] = \\beta_3+\\omega_3\\cdot\\mbox{PReLU}\\Bigl[\\gamma, \\beta_2+\\omega_2\\cdot\\mbox{PReLU}\\bigl[\\gamma, \\beta_1+\\omega_1\\cdot\\mbox{PReLU}[\\gamma, \\beta_0+\\omega_0x]\\bigr]\\Bigr],\n",
+        "\\end{equation}\n",
+        "\n",
+        "with parameters $\\boldsymbol\\phi=\\{\\beta_0,\\omega_0,\\beta_1,\\omega_1,\\beta_2,\\omega_2,\\beta_3,\\omega_3\\}$, where\n",
+        "\n",
+        "\\begin{equation}\n",
+        "\\mbox{PReLU}[\\gamma, z] = \\begin{cases} \\gamma\\cdot z & \\quad z \\leq0 \\\\ z & \\quad z> 0\\end{cases}.\n",
+        "\\end{equation}\n",
+        "\n",
+        "Suppose that we have a binary cross-entropy loss function (equation 5.20 from the book):\n",
+        "\n",
+        "\\begin{equation*}\n",
+        "\\ell_i  = -(1-y_{i})\\log\\Bigl[1-\\mbox{sig}[\\mbox{f}[\\mathbf{x}_i,\\boldsymbol\\phi]]\\Bigr] - y_{i}\\log\\Bigl[\\mbox{sig}[\\mbox{f}[\\mathbf{x}_i,\\boldsymbol\\phi]]\\Bigr].\n",
+        "\\end{equation*}\n",
+        "\n",
+        "Assume that we know the current values of $\\beta_{0},\\beta_{1},\\beta_{2},\\beta_{3},\\omega_{0},\\omega_{1},\\omega_{2},\\omega_{3}$, $\\gamma$, $x_i$ and $y_i$. We want to know how $\\ell_i$ changes when we make a small change to $\\beta_{0},\\beta_{1},\\beta_{2},\\beta_{3},\\omega_{0},\\omega_{1},\\omega_{2}$, or $\\omega_{3}$.  In other words, we want to compute the eight derivatives:\n",
+        "\n",
+        "\\begin{eqnarray*}\n",
+        "\\frac{\\partial \\ell_i}{\\partial \\beta_{0}}, \\quad \\frac{\\partial \\ell_i}{\\partial \\beta_{1}}, \\quad \\frac{\\partial \\ell_i}{\\partial \\beta_{2}}, \\quad \\frac{\\partial \\ell_i }{\\partial \\beta_{3}},  \\quad \\frac{\\partial \\ell_i}{\\partial \\omega_{0}}, \\quad \\frac{\\partial \\ell_i}{\\partial \\omega_{1}}, \\quad \\frac{\\partial \\ell_i}{\\partial \\omega_{2}},  \\quad\\mbox{and} \\quad \\frac{\\partial \\ell_i}{\\partial \\omega_{3}}.\n",
+        "\\end{eqnarray*}"
+      ],
+      "metadata": {
+        "id": "1DmMo2w63CmT"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# import library\n",
+        "import numpy as np"
+      ],
+      "metadata": {
+        "id": "RIPaoVN834Lj"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "Let's first define the original function and the loss term:"
+      ],
+      "metadata": {
+        "id": "32-ufWhc3v2c"
+      }
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "AakK_qen3BpU"
+      },
+      "outputs": [],
+      "source": [
+        "# Defines the activation function\n",
+        "def paramReLU(gamma,x):\n",
+        "  if x > 0:\n",
+        "    return x\n",
+        "  else:\n",
+        "    return x * gamma\n",
+        "\n",
+        "# Defines the main function\n",
+        "def fn(x, beta0, beta1, beta2, beta3, omega0, omega1, omega2, omega3, gamma):\n",
+        "  return beta3+omega3 * paramReLU(gamma, beta2 + omega2 * paramReLU(gamma, beta1 + omega1 * paramReLU(gamma, beta0 + omega0 * x)))\n",
+        "\n",
+        "# Logistic sigmoid\n",
+        "def sig(z):\n",
+        "  return 1./(1+np.exp(-z))\n",
+        "\n",
+        "# The loss function (equation 5.20 from book)\n",
+        "def loss(f,y):\n",
+        "  sig_net_out = sig(f)\n",
+        "  l = -(1-y) * np.log(1-sig_net_out) - y * np.log(sig_net_out)\n",
+        "  return l"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "Now we'll choose some values for the betas and the omegas and x and compute the output of the function:"
+      ],
+      "metadata": {
+        "id": "y7tf0ZMt5OXt"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "beta0 = 1.0; beta1 = -2.0; beta2 = -3.0; beta3 = 0.4\n",
+        "omega0 = 0.1; omega1 = -0.4; omega2 = 2.0; omega3 = -3.0\n",
+        "gamma = 0.2\n",
+        "x = 2.3; y =1.0\n",
+        "f_val = fn(x,beta0,beta1,beta2,beta3,omega0,omega1,omega2,omega3, gamma)\n",
+        "l_i_func = loss(f_val, y)\n",
+        "print('Loss full function = %3.3f'%l_i_func)"
+      ],
+      "metadata": {
+        "id": "pwvOcCxr41X_"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "# Forward pass\n",
+        "\n",
+        "We compute a series of intermediate values $f_0, h_0, f_1, h_1, f_2, h_2, f_3$, and finally the loss $\\ell$"
+      ],
+      "metadata": {
+        "id": "W6ZP62T5fU64"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "x = 2.3; y =1.0\n",
+        "gamma = 0.2\n",
+        "# Compute all the f_k and h_k terms\n",
+        "# I've done the first two for you\n",
+        "f0 = beta0+omega0 * x\n",
+        "h1 = paramReLU(gamma, f0)\n",
+        "\n",
+        "\n",
+        "# TODO:  Replace the code below\n",
+        "f1 = 0\n",
+        "h2 = 0\n",
+        "f2 = 0\n",
+        "h3 = 0\n",
+        "f3 = 0\n",
+        "\n",
+        "\n",
+        "# Compute the loss and print\n",
+        "# The answer should be the same as when we computed the full function above\n",
+        "l_i = loss(f3, y)\n",
+        "print(\"Loss forward pass = %3.3f\"%(l_i))\n"
+      ],
+      "metadata": {
+        "id": "z-BckTpMf5PL"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "# Backward pass:  Derivative of loss function with respect to function output\n",
+        "\n",
+        "Now, we'll compute the derivative $\\frac{dl}{df_3}$ of the loss function with respect to the network output $f_3$.  In other words, we are asking how does the loss change as we make a small change in the network output.\n",
+        "\n",
+        "Since the loss it itself a function of $\\mbox{sig}[f_3]$ we'll compute this using the chain rule:\n",
+        "\n",
+        "\\begin{equation}\n",
+        "\\frac{dl}{df_3} = \\frac{d\\mbox{sig}[f_3]}{df_3}\\cdot \\frac{dl}{d\\mbox{sig}[f_3]}\n",
+        "\\end{equation}\n",
+        "\n",
+        "Your job is to compute the two quantities on the right hand side.\n"
+      ],
+      "metadata": {
+        "id": "TbFbxz64Xz4I"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Compute the derivative of the loss with respect to the function output f_val\n",
+        "def dl_df(f_val,y):\n",
+        "  # Compute sigmoid of network output\n",
+        "  sig_f_val = sig(f_val)\n",
+        "  # Compute the derivative of loss with respect to network output using chain rule\n",
+        "  dl_df_val = dsig_df(f_val) * dl_dsigf(sig_f_val, y)\n",
+        "  # Return the derivative\n",
+        "  return dl_df_val"
+      ],
+      "metadata": {
+        "id": "ZWKAq6HC90qV"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# MOODLE ANSWER # Notebook V 1a: Copy this code when you have finished it.\n",
+        "\n",
+        "# Compute the derivative of the logistic sigmoid function with respect to its input (as a closed form solution)\n",
+        "def dsig_df(f_val):\n",
+        "  # TODO Write this function\n",
+        "  # Replace this line:\n",
+        "  return 1\n",
+        "\n",
+        "# Compute the derivative of the loss with respect to the logistic sigmoid (as a closed form solution)\n",
+        "def dl_dsigf(sig_f_val, y):\n",
+        "  # TODO Write this function\n",
+        "  # Replace this line:\n",
+        "  return 1"
+      ],
+      "metadata": {
+        "id": "lIngYAgPq-5I"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "Let's run that for some f_val, y.  Check previous practicals to see how you can check whether your answer is correct."
+      ],
+      "metadata": {
+        "id": "Q-j-i8khXzbK"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "y = 0.0\n",
+        "dl_df3 = dl_df(f3,y)\n",
+        "print(\"Moodle Answer Notebook V 1b: dldh3=%3.3f\"%(dl_df3))\n",
+        "\n",
+        "y= 1.0\n",
+        "dl_df3 = dl_df(f3,y)\n",
+        "print(\"Moodle Answer Notebook V 1c: dldh3=%3.3f\"%(dl_df3))"
+      ],
+      "metadata": {
+        "id": "Z7Lb5BibY50H"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "# Backward pass:  Derivative of activation function with respect to preactivations\n",
+        "\n",
+        "Write a function to compute the derivative $\\frac{\\partial h}{\\partial f}$ of the activation function (parametric ReLU) with respect to its input.\n"
+      ],
+      "metadata": {
+        "id": "BA7QaOzejzZw"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# MOODLE ANSWER Notebook V 2a: Copy this code when you have finished it.\n",
+        "\n",
+        "def dh_df(gamma, f_val):\n",
+        "  # TODO:  Write this function\n",
+        "  # Replace this line:\n",
+        "  return 1\n"
+      ],
+      "metadata": {
+        "id": "bBPfPg04j-Qw"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "Let's run that for some values of f_val.  Check previous practicals to see how you can check whether your answer is correct."
+      ],
+      "metadata": {
+        "id": "QRNCM0EGk9-w"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "f_val_test = 0.6\n",
+        "dh_df_val = dh_df(gamma, f_val_test)\n",
+        "print(\"Moodle Answer Notebook V 2b: dhdf=%3.3f\"%(dh_df_val))\n",
+        "\n",
+        "f_val_test = -0.4\n",
+        "dh_df_val = dh_df(gamma, f_val_test)\n",
+        "print(\"Moodle Answer Notebook V 2c: dhdf=%3.3f\"%(dh_df_val))"
+      ],
+      "metadata": {
+        "id": "bql8VZIGk8Wy"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        " # Backward pass:  Compute the derivatives of $l_i$ with respect to the intermediate quantities but in reverse order:\n",
+        "\n",
+        "\\begin{eqnarray}\n",
+        "\\frac{\\partial \\ell_i}{\\partial h_3}, \\quad \\frac{\\partial \\ell_i}{\\partial f_2}, \\quad\n",
+        "\\frac{\\partial \\ell_i}{\\partial h_2}, \\quad \\frac{\\partial \\ell_i}{\\partial f_1}, \\quad\\frac{\\partial \\ell_i}{\\partial h_1},  \\quad\\mbox{and} \\quad \\frac{\\partial \\ell_i}{\\partial f_0}.\n",
+        "\\end{eqnarray}\n",
+        "\n",
+        "The first of these derivatives can be calculated using the chain rule:\n",
+        "\n",
+        "\\begin{equation}\n",
+        "\\frac{\\partial \\ell_i}{\\partial h_{3}} =\\frac{\\partial f_{3}}{\\partial h_{3}} \\frac{\\partial \\ell_i}{\\partial f_{3}} .\n",
+        "\\end{equation}\n",
+        "\n",
+        "The left-hand side asks how $\\ell_i$ changes when $h_{3}$ changes.  The right-hand side says we can decompose this into (i) how $\\ell_i$ changes when $f_{3}$ changes and how $f_{3}$ changes when $h_{3}$ changes.  So you get a chain of events happening:  $h_{3}$ changes $f_{3}$, which changes $\\ell_i$, and the derivatives represent the effects of this chain.  Notice that we computed the first of these derivatives already.  The second term is the derivative of $\\beta_{3} + \\omega_{3}h_{3}$ with respect to $h_3$ which is simply $\\omega_3$.  \n",
+        "\n",
+        "We can continue in this way, computing the derivatives of the output with respect to these intermediate quantities:\n",
+        "\n",
+        "\\begin{eqnarray}\n",
+        "\\frac{\\partial \\ell_i}{\\partial f_{2}} &=& \\frac{\\partial h_{3}}{\\partial f_{2}}\\left(\n",
+        "\\frac{\\partial f_{3}}{\\partial h_{3}}\\frac{\\partial \\ell_i}{\\partial f_{3}} \\right)\n",
+        "\\nonumber \\\\\n",
+        "\\frac{\\partial \\ell_i}{\\partial h_{2}} &=& \\frac{\\partial f_{2}}{\\partial h_{2}}\\left(\\frac{\\partial h_{3}}{\\partial f_{2}}\\frac{\\partial f_{3}}{\\partial h_{3}}\\frac{\\partial \\ell_i}{\\partial f_{3}}\\right)\\nonumber \\\\\n",
+        "\\frac{\\partial \\ell_i}{\\partial f_{1}} &=& \\frac{\\partial h_{2}}{\\partial f_{1}}\\left( \\frac{\\partial f_{2}}{\\partial h_{2}}\\frac{\\partial h_{3}}{\\partial f_{2}}\\frac{\\partial f_{3}}{\\partial h_{3}}\\frac{\\partial \\ell_i}{\\partial f_{3}} \\right)\\nonumber \\\\\n",
+        "\\frac{\\partial \\ell_i}{\\partial h_{1}} &=& \\frac{\\partial f_{1}}{\\partial h_{1}}\\left(\\frac{\\partial h_{2}}{\\partial f_{1}} \\frac{\\partial f_{2}}{\\partial h_{2}}\\frac{\\partial h_{3}}{\\partial f_{2}}\\frac{\\partial f_{3}}{\\partial h_{3}}\\frac{\\partial \\ell_i}{\\partial f_{3}} \\right)\\nonumber \\\\\n",
+        "\\frac{\\partial \\ell_i}{\\partial f_{0}} &=& \\frac{\\partial h_{1}}{\\partial f_{0}}\\left(\\frac{\\partial f_{1}}{\\partial h_{1}}\\frac{\\partial h_{2}}{\\partial f_{1}} \\frac{\\partial f_{2}}{\\partial h_{2}}\\frac{\\partial h_{3}}{\\partial f_{2}}\\frac{\\partial f_{3}}{\\partial h_{3}}\\frac{\\partial \\ell_i}{\\partial f_{3}} \\right).\n",
+        "\\end{eqnarray}\n",
+        "\n",
+        "In each case, we have already computed all of the terms except the last one in the previous step, and the last term is simple to evaluate.  This is called the **backward pass**."
+      ],
+      "metadata": {
+        "id": "jay8NYWdFHuZ"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "x = 2.3; y =1.0\n",
+        "dldf3 = dl_df(f3,y)"
+      ],
+      "metadata": {
+        "id": "RSC_2CIfKF1b"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# MOODLE ANSWER Notebook V 3a: Copy this code when you have finished it.\n",
+        "# TODO -- Compute the derivatives of the output with respect\n",
+        "# to the intermediate computations h_k and f_k (i.e, run the backward pass)\n",
+        "# I've done the first two for you.  You replace the code below:\n",
+        "# Replace the code below\n",
+        "dldh3 = 1\n",
+        "dldf2 = 1\n",
+        "dldh2 = 1\n",
+        "dldf1 = 1\n",
+        "dldh1 = 1\n",
+        "dldf0 = 1"
+      ],
+      "metadata": {
+        "id": "gCQJeI--Egdl"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "Finally, we consider how the loss~$\\ell_{i}$ changes when we change the parameters $\\beta_{\\bullet}$ and $\\omega_{\\bullet}$. Once more, we apply the chain rule:\n",
+        "\n",
+        "\n",
+        "\n",
+        "\n",
+        "\\begin{eqnarray}\n",
+        "\\frac{\\partial \\ell_i}{\\partial \\beta_{k}} &=& \\frac{\\partial f_{k}}{\\partial \\beta_{k}}\\frac{\\partial \\ell_i}{\\partial f_{k}}\\nonumber \\\\\n",
+        "\\frac{\\partial \\ell_i}{\\partial \\omega_{k}} &=& \\frac{\\partial f_{k}}{\\partial \\omega_{k}}\\frac{\\partial \\ell_i}{\\partial f_{k}}.\n",
+        "\\end{eqnarray}\n",
+        "\n",
+        "\\noindent In each case, the second term on the right-hand side was computed in step 2. When $k>0$, we have~$f_{k}=\\beta_{k}+\\omega_k \\cdot h_{k}$, so:\n",
+        "\n",
+        "\\begin{eqnarray}\n",
+        "\\frac{\\partial f_{k}}{\\partial \\beta_{k}} = 1 \\quad\\quad\\mbox{and}\\quad \\quad \\frac{\\partial f_{k}}{\\partial \\omega_{k}} &=& h_{k}.\n",
+        "\\end{eqnarray}"
+      ],
+      "metadata": {
+        "id": "FlzlThQPGpkU"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# MOODLE ANSWER Notebook V 3b: Copy this code when you have finished it.\n",
+        "# TODO -- Calculate the final derivatives with respect to the beta and omega terms\n",
+        "# Replace these terms\n",
+        "dldbeta3 = 1\n",
+        "dldomega3 = 1\n",
+        "dldbeta2 = 1\n",
+        "dldomega2 = 1\n",
+        "dldbeta1 = 1\n",
+        "dldomega1 = 1\n",
+        "dldbeta0 = 1\n",
+        "dldomega0 = 1"
+      ],
+      "metadata": {
+        "id": "1I2BhqZhGMK6"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Print the last two values out (enter these into Moodle).  Again, think about how you can test whether these are correct.\n",
+        "print('Moodle Answer Notebook V 3c: dldbeta0=%3.3f'%(dldbeta0))\n",
+        "print('Moodle Answer Notebook V 3d: dldOmega0=%3.3f'%(dldomega0))"
+      ],
+      "metadata": {
+        "id": "38eiOn2aHgHI"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "# Compute the derivatives of  $\\ell_i$  with respect to the parmeter $\\gamma$ of the parametric ReLU function.  \n",
+        "\n",
+        "In other words, compute:\n",
+        "\n",
+        "\\begin{equation}\n",
+        "\\frac{d\\ell_i}{d\\gamma}\n",
+        "\\end{equation}\n",
+        "\n",
+        "Along the way, we will need to compute derivatives\n",
+        "\n",
+        "\\begin{equation}\n",
+        "\\frac{dh_k(\\gamma,f_{k-1})}{d\\gamma}\n",
+        "\\end{equation}\n",
+        "\n",
+        "This is quite difficult and not worth many marks, so don't spend too much time on it if you are confused!"
+      ],
+      "metadata": {
+        "id": "lhD5AoUHx3DS"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Computes how an activation changes with a small change in gamma assuming preactivations are f\n",
+        "# MOODLE ANSWER # Notebook V 4a: Copy this code when you have finished it.\n",
+        "def dhdgamma(gamma, f):\n",
+        "  # TODO -- Write this function\n",
+        "  # Replace this line\n",
+        "  return 1"
+      ],
+      "metadata": {
+        "id": "yC-9MTQevliP"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Compute how the loss changes with gamma\n",
+        "# Replace this line:\n",
+        "# MOODLE ANSWER # Notebook V 4b: Copy this code when you have finished it.\n",
+        "dldgamma = 1"
+      ],
+      "metadata": {
+        "id": "DiNQrveoLuHR"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "print(\"Moodle Answer Notebook V 4c: dldgamma = %3.3f\"%(dldgamma))"
+      ],
+      "metadata": {
+        "id": "YHxmAEnxzy3O"
+      },
+      "execution_count": null,
+      "outputs": []
+    }
+  ]
+}

LICENSE

@@ -1,346 +1,346 @@
-Creative Commons Attribution-NonCommercial-NoDerivatives 4.0
-International Public License
-
-By exercising the Licensed Rights (defined below), You accept and agree
-to be bound by the terms and conditions of this Creative Commons
-Attribution-NonCommercial-NoDerivatives 4.0 International Public
-License ("Public License"). To the extent this Public License may be
-interpreted as a contract, You are granted the Licensed Rights in
-consideration of Your acceptance of these terms and conditions, and the
-Licensor grants You such rights in consideration of benefits the
-Licensor receives from making the Licensed Material available under
-these terms and conditions.
-
-
-Section 1 -- Definitions.
-
-  a. Adapted Material means material subject to Copyright and Similar
-     Rights that is derived from or based upon the Licensed Material
-     and in which the Licensed Material is translated, altered,
-     arranged, transformed, or otherwise modified in a manner requiring
-     permission under the Copyright and Similar Rights held by the
-     Licensor. For purposes of this Public License, where the Licensed
-     Material is a musical work, performance, or sound recording,
-     Adapted Material is always produced where the Licensed Material is
-     synched in timed relation with a moving image.
-
-  b. Copyright and Similar Rights means copyright and/or similar rights
-     closely related to copyright including, without limitation,
-     performance, broadcast, sound recording, and Sui Generis Database
-     Rights, without regard to how the rights are labeled or
-     categorized. For purposes of this Public License, the rights
-     specified in Section 2(b)(1)-(2) are not Copyright and Similar
-     Rights.
-
-  c. Effective Technological Measures means those measures that, in the
-     absence of proper authority, may not be circumvented under laws
-     fulfilling obligations under Article 11 of the WIPO Copyright
-     Treaty adopted on December 20, 1996, and/or similar international
-     agreements.
-
-  d. Exceptions and Limitations means fair use, fair dealing, and/or
-     any other exception or limitation to Copyright and Similar Rights
-     that applies to Your use of the Licensed Material.
-
-  e. Licensed Material means the artistic or literary work, database,
-     or other material to which the Licensor applied this Public
-     License.
-
-  f. Licensed Rights means the rights granted to You subject to the
-     terms and conditions of this Public License, which are limited to
-     all Copyright and Similar Rights that apply to Your use of the
-     Licensed Material and that the Licensor has authority to license.
-
-  g. Licensor means the individual(s) or entity(ies) granting rights
-     under this Public License.
-
-  h. NonCommercial means not primarily intended for or directed towards
-     commercial advantage or monetary compensation. For purposes of
-     this Public License, the exchange of the Licensed Material for
-     other material subject to Copyright and Similar Rights by digital
-     file-sharing or similar means is NonCommercial provided there is
-     no payment of monetary compensation in connection with the
-     exchange.
-
-  i. Share means to provide material to the public by any means or
-     process that requires permission under the Licensed Rights, such
-     as reproduction, public display, public performance, distribution,
-     dissemination, communication, or importation, and to make material
-     available to the public including in ways that members of the
-     public may access the material from a place and at a time
-     individually chosen by them.
-
-  j. Sui Generis Database Rights means rights other than copyright
-     resulting from Directive 96/9/EC of the European Parliament and of
-     the Council of 11 March 1996 on the legal protection of databases,
-     as amended and/or succeeded, as well as other essentially
-     equivalent rights anywhere in the world.
-
-  k. You means the individual or entity exercising the Licensed Rights
-     under this Public License. Your has a corresponding meaning.
-
-
-Section 2 -- Scope.
-
-  a. License grant.
-
-       1. Subject to the terms and conditions of this Public License,
-          the Licensor hereby grants You a worldwide, royalty-free,
-          non-sublicensable, non-exclusive, irrevocable license to
-          exercise the Licensed Rights in the Licensed Material to:
-
-            a. reproduce and Share the Licensed Material, in whole or
-               in part, for NonCommercial purposes only; and
-
-            b. produce and reproduce, but not Share, Adapted Material
-               for NonCommercial purposes only.
-
-       2. Exceptions and Limitations. For the avoidance of doubt, where
-          Exceptions and Limitations apply to Your use, this Public
-          License does not apply, and You do not need to comply with
-          its terms and conditions.
-
-       3. Term. The term of this Public License is specified in Section
-          6(a).
-
-       4. Media and formats; technical modifications allowed. The
-          Licensor authorizes You to exercise the Licensed Rights in
-          all media and formats whether now known or hereafter created,
-          and to make technical modifications necessary to do so. The
-          Licensor waives and/or agrees not to assert any right or
-          authority to forbid You from making technical modifications
-          necessary to exercise the Licensed Rights, including
-          technical modifications necessary to circumvent Effective
-          Technological Measures. For purposes of this Public License,
-          simply making modifications authorized by this Section 2(a)
-          (4) never produces Adapted Material.
-
-       5. Downstream recipients.
-
-            a. Offer from the Licensor -- Licensed Material. Every
-               recipient of the Licensed Material automatically
-               receives an offer from the Licensor to exercise the
-               Licensed Rights under the terms and conditions of this
-               Public License.
-
-            b. No downstream restrictions. You may not offer or impose
-               any additional or different terms or conditions on, or
-               apply any Effective Technological Measures to, the
-               Licensed Material if doing so restricts exercise of the
-               Licensed Rights by any recipient of the Licensed
-               Material.
-
-       6. No endorsement. Nothing in this Public License constitutes or
-          may be construed as permission to assert or imply that You
-          are, or that Your use of the Licensed Material is, connected
-          with, or sponsored, endorsed, or granted official status by,
-          the Licensor or others designated to receive attribution as
-          provided in Section 3(a)(1)(A)(i).
-
-  b. Other rights.
-
-       1. Moral rights, such as the right of integrity, are not
-          licensed under this Public License, nor are publicity,
-          privacy, and/or other similar personality rights; however, to
-          the extent possible, the Licensor waives and/or agrees not to
-          assert any such rights held by the Licensor to the limited
-          extent necessary to allow You to exercise the Licensed
-          Rights, but not otherwise.
-
-       2. Patent and trademark rights are not licensed under this
-          Public License.
-
-       3. To the extent possible, the Licensor waives any right to
-          collect royalties from You for the exercise of the Licensed
-          Rights, whether directly or through a collecting society
-          under any voluntary or waivable statutory or compulsory
-          licensing scheme. In all other cases the Licensor expressly
-          reserves any right to collect such royalties, including when
-          the Licensed Material is used other than for NonCommercial
-          purposes.
-
-
-Section 3 -- License Conditions.
-
-Your exercise of the Licensed Rights is expressly made subject to the
-following conditions.
-
-  a. Attribution.
-
-       1. If You Share the Licensed Material, You must:
-
-            a. retain the following if it is supplied by the Licensor
-               with the Licensed Material:
-
-                 i. identification of the creator(s) of the Licensed
-                    Material and any others designated to receive
-                    attribution, in any reasonable manner requested by
-                    the Licensor (including by pseudonym if
-                    designated);
-
-                ii. a copyright notice;
-
-               iii. a notice that refers to this Public License;
-
-                iv. a notice that refers to the disclaimer of
-                    warranties;
-
-                 v. a URI or hyperlink to the Licensed Material to the
-                    extent reasonably practicable;
-
-            b. indicate if You modified the Licensed Material and
-               retain an indication of any previous modifications; and
-
-            c. indicate the Licensed Material is licensed under this
-               Public License, and include the text of, or the URI or
-               hyperlink to, this Public License.
-
-          For the avoidance of doubt, You do not have permission under
-          this Public License to Share Adapted Material.
-
-       2. You may satisfy the conditions in Section 3(a)(1) in any
-          reasonable manner based on the medium, means, and context in
-          which You Share the Licensed Material. For example, it may be
-          reasonable to satisfy the conditions by providing a URI or
-          hyperlink to a resource that includes the required
-          information.
-
-       3. If requested by the Licensor, You must remove any of the
-          information required by Section 3(a)(1)(A) to the extent
-          reasonably practicable.
-
-
-Section 4 -- Sui Generis Database Rights.
-
-Where the Licensed Rights include Sui Generis Database Rights that
-apply to Your use of the Licensed Material:
-
-  a. for the avoidance of doubt, Section 2(a)(1) grants You the right
-     to extract, reuse, reproduce, and Share all or a substantial
-     portion of the contents of the database for NonCommercial purposes
-     only and provided You do not Share Adapted Material;
-
-  b. if You include all or a substantial portion of the database
-     contents in a database in which You have Sui Generis Database
-     Rights, then the database in which You have Sui Generis Database
-     Rights (but not its individual contents) is Adapted Material; and
-
-  c. You must comply with the conditions in Section 3(a) if You Share
-     all or a substantial portion of the contents of the database.
-
-For the avoidance of doubt, this Section 4 supplements and does not
-replace Your obligations under this Public License where the Licensed
-Rights include other Copyright and Similar Rights.
-
-
-Section 5 -- Disclaimer of Warranties and Limitation of Liability.
-
-  a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE
-     EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS
-     AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF
-     ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS,
-     IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION,
-     WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR
-     PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS,
-     ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT
-     KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT
-     ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU.
-
-  b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE
-     TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION,
-     NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT,
-     INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES,
-     COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR
-     USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN
-     ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR
-     DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR
-     IN PART, THIS LIMITATION MAY NOT APPLY TO YOU.
-
-  c. The disclaimer of warranties and limitation of liability provided
-     above shall be interpreted in a manner that, to the extent
-     possible, most closely approximates an absolute disclaimer and
-     waiver of all liability.
-
-
-Section 6 -- Term and Termination.
-
-  a. This Public License applies for the term of the Copyright and
-     Similar Rights licensed here. However, if You fail to comply with
-     this Public License, then Your rights under this Public License
-     terminate automatically.
-
-  b. Where Your right to use the Licensed Material has terminated under
-     Section 6(a), it reinstates:
-
-       1. automatically as of the date the violation is cured, provided
-          it is cured within 30 days of Your discovery of the
-          violation; or
-
-       2. upon express reinstatement by the Licensor.
-
-     For the avoidance of doubt, this Section 6(b) does not affect any
-     right the Licensor may have to seek remedies for Your violations
-     of this Public License.
-
-  c. For the avoidance of doubt, the Licensor may also offer the
-     Licensed Material under separate terms or conditions or stop
-     distributing the Licensed Material at any time; however, doing so
-     will not terminate this Public License.
-
-  d. Sections 1, 5, 6, 7, and 8 survive termination of this Public
-     License.
-
-
-Section 7 -- Other Terms and Conditions.
-
-  a. The Licensor shall not be bound by any additional or different
-     terms or conditions communicated by You unless expressly agreed.
-
-  b. Any arrangements, understandings, or agreements regarding the
-     Licensed Material not stated herein are separate from and
-     independent of the terms and conditions of this Public License.
-
-
-Section 8 -- Interpretation.
-
-  a. For the avoidance of doubt, this Public License does not, and
-     shall not be interpreted to, reduce, limit, restrict, or impose
-     conditions on any use of the Licensed Material that could lawfully
-     be made without permission under this Public License.
-
-  b. To the extent possible, if any provision of this Public License is
-     deemed unenforceable, it shall be automatically reformed to the
-     minimum extent necessary to make it enforceable. If the provision
-     cannot be reformed, it shall be severed from this Public License
-     without affecting the enforceability of the remaining terms and
-     conditions.
-
-  c. No term or condition of this Public License will be waived and no
-     failure to comply consented to unless expressly agreed to by the
-     Licensor.
-
-  d. Nothing in this Public License constitutes or may be interpreted
-     as a limitation upon, or waiver of, any privileges and immunities
-     that apply to the Licensor or You, including from the legal
-     processes of any jurisdiction or authority.
-
-=======================================================================
-
-Creative Commons is not a party to its public
-licenses. Notwithstanding, Creative Commons may elect to apply one of
-its public licenses to material it publishes and in those instances
-will be considered the “Licensor.” The text of the Creative Commons
-public licenses is dedicated to the public domain under the CC0 Public
-Domain Dedication. Except for the limited purpose of indicating that
-material is shared under a Creative Commons public license or as
-otherwise permitted by the Creative Commons policies published at
-creativecommons.org/policies, Creative Commons does not authorize the
-use of the trademark "Creative Commons" or any other trademark or logo
-of Creative Commons without its prior written consent including,
-without limitation, in connection with any unauthorized modifications
-to any of its public licenses or any other arrangements,
-understandings, or agreements concerning use of licensed material. For
-the avoidance of doubt, this paragraph does not form part of the
-public licenses.
-
-Creative Commons may be contacted at creativecommons.org.
+Creative Commons Attribution-NonCommercial-NoDerivatives 4.0
+International Public License
+
+By exercising the Licensed Rights (defined below), You accept and agree
+to be bound by the terms and conditions of this Creative Commons
+Attribution-NonCommercial-NoDerivatives 4.0 International Public
+License ("Public License"). To the extent this Public License may be
+interpreted as a contract, You are granted the Licensed Rights in
+consideration of Your acceptance of these terms and conditions, and the
+Licensor grants You such rights in consideration of benefits the
+Licensor receives from making the Licensed Material available under
+these terms and conditions.
+
+
+Section 1 -- Definitions.
+
+  a. Adapted Material means material subject to Copyright and Similar
+     Rights that is derived from or based upon the Licensed Material
+     and in which the Licensed Material is translated, altered,
+     arranged, transformed, or otherwise modified in a manner requiring
+     permission under the Copyright and Similar Rights held by the
+     Licensor. For purposes of this Public License, where the Licensed
+     Material is a musical work, performance, or sound recording,
+     Adapted Material is always produced where the Licensed Material is
+     synched in timed relation with a moving image.
+
+  b. Copyright and Similar Rights means copyright and/or similar rights
+     closely related to copyright including, without limitation,
+     performance, broadcast, sound recording, and Sui Generis Database
+     Rights, without regard to how the rights are labeled or
+     categorized. For purposes of this Public License, the rights
+     specified in Section 2(b)(1)-(2) are not Copyright and Similar
+     Rights.
+
+  c. Effective Technological Measures means those measures that, in the
+     absence of proper authority, may not be circumvented under laws
+     fulfilling obligations under Article 11 of the WIPO Copyright
+     Treaty adopted on December 20, 1996, and/or similar international
+     agreements.
+
+  d. Exceptions and Limitations means fair use, fair dealing, and/or
+     any other exception or limitation to Copyright and Similar Rights
+     that applies to Your use of the Licensed Material.
+
+  e. Licensed Material means the artistic or literary work, database,
+     or other material to which the Licensor applied this Public
+     License.
+
+  f. Licensed Rights means the rights granted to You subject to the
+     terms and conditions of this Public License, which are limited to
+     all Copyright and Similar Rights that apply to Your use of the
+     Licensed Material and that the Licensor has authority to license.
+
+  g. Licensor means the individual(s) or entity(ies) granting rights
+     under this Public License.
+
+  h. NonCommercial means not primarily intended for or directed towards
+     commercial advantage or monetary compensation. For purposes of
+     this Public License, the exchange of the Licensed Material for
+     other material subject to Copyright and Similar Rights by digital
+     file-sharing or similar means is NonCommercial provided there is
+     no payment of monetary compensation in connection with the
+     exchange.
+
+  i. Share means to provide material to the public by any means or
+     process that requires permission under the Licensed Rights, such
+     as reproduction, public display, public performance, distribution,
+     dissemination, communication, or importation, and to make material
+     available to the public including in ways that members of the
+     public may access the material from a place and at a time
+     individually chosen by them.
+
+  j. Sui Generis Database Rights means rights other than copyright
+     resulting from Directive 96/9/EC of the European Parliament and of
+     the Council of 11 March 1996 on the legal protection of databases,
+     as amended and/or succeeded, as well as other essentially
+     equivalent rights anywhere in the world.
+
+  k. You means the individual or entity exercising the Licensed Rights
+     under this Public License. Your has a corresponding meaning.
+
+
+Section 2 -- Scope.
+
+  a. License grant.
+
+       1. Subject to the terms and conditions of this Public License,
+          the Licensor hereby grants You a worldwide, royalty-free,
+          non-sublicensable, non-exclusive, irrevocable license to
+          exercise the Licensed Rights in the Licensed Material to:
+
+            a. reproduce and Share the Licensed Material, in whole or
+               in part, for NonCommercial purposes only; and
+
+            b. produce and reproduce, but not Share, Adapted Material
+               for NonCommercial purposes only.
+
+       2. Exceptions and Limitations. For the avoidance of doubt, where
+          Exceptions and Limitations apply to Your use, this Public
+          License does not apply, and You do not need to comply with
+          its terms and conditions.
+
+       3. Term. The term of this Public License is specified in Section
+          6(a).
+
+       4. Media and formats; technical modifications allowed. The
+          Licensor authorizes You to exercise the Licensed Rights in
+          all media and formats whether now known or hereafter created,
+          and to make technical modifications necessary to do so. The
+          Licensor waives and/or agrees not to assert any right or
+          authority to forbid You from making technical modifications
+          necessary to exercise the Licensed Rights, including
+          technical modifications necessary to circumvent Effective
+          Technological Measures. For purposes of this Public License,
+          simply making modifications authorized by this Section 2(a)
+          (4) never produces Adapted Material.
+
+       5. Downstream recipients.
+
+            a. Offer from the Licensor -- Licensed Material. Every
+               recipient of the Licensed Material automatically
+               receives an offer from the Licensor to exercise the
+               Licensed Rights under the terms and conditions of this
+               Public License.
+
+            b. No downstream restrictions. You may not offer or impose
+               any additional or different terms or conditions on, or
+               apply any Effective Technological Measures to, the
+               Licensed Material if doing so restricts exercise of the
+               Licensed Rights by any recipient of the Licensed
+               Material.
+
+       6. No endorsement. Nothing in this Public License constitutes or
+          may be construed as permission to assert or imply that You
+          are, or that Your use of the Licensed Material is, connected
+          with, or sponsored, endorsed, or granted official status by,
+          the Licensor or others designated to receive attribution as
+          provided in Section 3(a)(1)(A)(i).
+
+  b. Other rights.
+
+       1. Moral rights, such as the right of integrity, are not
+          licensed under this Public License, nor are publicity,
+          privacy, and/or other similar personality rights; however, to
+          the extent possible, the Licensor waives and/or agrees not to
+          assert any such rights held by the Licensor to the limited
+          extent necessary to allow You to exercise the Licensed
+          Rights, but not otherwise.
+
+       2. Patent and trademark rights are not licensed under this
+          Public License.
+
+       3. To the extent possible, the Licensor waives any right to
+          collect royalties from You for the exercise of the Licensed
+          Rights, whether directly or through a collecting society
+          under any voluntary or waivable statutory or compulsory
+          licensing scheme. In all other cases the Licensor expressly
+          reserves any right to collect such royalties, including when
+          the Licensed Material is used other than for NonCommercial
+          purposes.
+
+
+Section 3 -- License Conditions.
+
+Your exercise of the Licensed Rights is expressly made subject to the
+following conditions.
+
+  a. Attribution.
+
+       1. If You Share the Licensed Material, You must:
+
+            a. retain the following if it is supplied by the Licensor
+               with the Licensed Material:
+
+                 i. identification of the creator(s) of the Licensed
+                    Material and any others designated to receive
+                    attribution, in any reasonable manner requested by
+                    the Licensor (including by pseudonym if
+                    designated);
+
+                ii. a copyright notice;
+
+               iii. a notice that refers to this Public License;
+
+                iv. a notice that refers to the disclaimer of
+                    warranties;
+
+                 v. a URI or hyperlink to the Licensed Material to the
+                    extent reasonably practicable;
+
+            b. indicate if You modified the Licensed Material and
+               retain an indication of any previous modifications; and
+
+            c. indicate the Licensed Material is licensed under this
+               Public License, and include the text of, or the URI or
+               hyperlink to, this Public License.
+
+          For the avoidance of doubt, You do not have permission under
+          this Public License to Share Adapted Material.
+
+       2. You may satisfy the conditions in Section 3(a)(1) in any
+          reasonable manner based on the medium, means, and context in
+          which You Share the Licensed Material. For example, it may be
+          reasonable to satisfy the conditions by providing a URI or
+          hyperlink to a resource that includes the required
+          information.
+
+       3. If requested by the Licensor, You must remove any of the
+          information required by Section 3(a)(1)(A) to the extent
+          reasonably practicable.
+
+
+Section 4 -- Sui Generis Database Rights.
+
+Where the Licensed Rights include Sui Generis Database Rights that
+apply to Your use of the Licensed Material:
+
+  a. for the avoidance of doubt, Section 2(a)(1) grants You the right
+     to extract, reuse, reproduce, and Share all or a substantial
+     portion of the contents of the database for NonCommercial purposes
+     only and provided You do not Share Adapted Material;
+
+  b. if You include all or a substantial portion of the database
+     contents in a database in which You have Sui Generis Database
+     Rights, then the database in which You have Sui Generis Database
+     Rights (but not its individual contents) is Adapted Material; and
+
+  c. You must comply with the conditions in Section 3(a) if You Share
+     all or a substantial portion of the contents of the database.
+
+For the avoidance of doubt, this Section 4 supplements and does not
+replace Your obligations under this Public License where the Licensed
+Rights include other Copyright and Similar Rights.
+
+
+Section 5 -- Disclaimer of Warranties and Limitation of Liability.
+
+  a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE
+     EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS
+     AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF
+     ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS,
+     IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION,
+     WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR
+     PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS,
+     ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT
+     KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT
+     ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU.
+
+  b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE
+     TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION,
+     NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT,
+     INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES,
+     COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR
+     USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN
+     ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR
+     DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR
+     IN PART, THIS LIMITATION MAY NOT APPLY TO YOU.
+
+  c. The disclaimer of warranties and limitation of liability provided
+     above shall be interpreted in a manner that, to the extent
+     possible, most closely approximates an absolute disclaimer and
+     waiver of all liability.
+
+
+Section 6 -- Term and Termination.
+
+  a. This Public License applies for the term of the Copyright and
+     Similar Rights licensed here. However, if You fail to comply with
+     this Public License, then Your rights under this Public License
+     terminate automatically.
+
+  b. Where Your right to use the Licensed Material has terminated under
+     Section 6(a), it reinstates:
+
+       1. automatically as of the date the violation is cured, provided
+          it is cured within 30 days of Your discovery of the
+          violation; or
+
+       2. upon express reinstatement by the Licensor.
+
+     For the avoidance of doubt, this Section 6(b) does not affect any
+     right the Licensor may have to seek remedies for Your violations
+     of this Public License.
+
+  c. For the avoidance of doubt, the Licensor may also offer the
+     Licensed Material under separate terms or conditions or stop
+     distributing the Licensed Material at any time; however, doing so
+     will not terminate this Public License.
+
+  d. Sections 1, 5, 6, 7, and 8 survive termination of this Public
+     License.
+
+
+Section 7 -- Other Terms and Conditions.
+
+  a. The Licensor shall not be bound by any additional or different
+     terms or conditions communicated by You unless expressly agreed.
+
+  b. Any arrangements, understandings, or agreements regarding the
+     Licensed Material not stated herein are separate from and
+     independent of the terms and conditions of this Public License.
+
+
+Section 8 -- Interpretation.
+
+  a. For the avoidance of doubt, this Public License does not, and
+     shall not be interpreted to, reduce, limit, restrict, or impose
+     conditions on any use of the Licensed Material that could lawfully
+     be made without permission under this Public License.
+
+  b. To the extent possible, if any provision of this Public License is
+     deemed unenforceable, it shall be automatically reformed to the
+     minimum extent necessary to make it enforceable. If the provision
+     cannot be reformed, it shall be severed from this Public License
+     without affecting the enforceability of the remaining terms and
+     conditions.
+
+  c. No term or condition of this Public License will be waived and no
+     failure to comply consented to unless expressly agreed to by the
+     Licensor.
+
+  d. Nothing in this Public License constitutes or may be interpreted
+     as a limitation upon, or waiver of, any privileges and immunities
+     that apply to the Licensor or You, including from the legal
+     processes of any jurisdiction or authority.
+
+=======================================================================
+
+Creative Commons is not a party to its public
+licenses. Notwithstanding, Creative Commons may elect to apply one of
+its public licenses to material it publishes and in those instances
+will be considered the “Licensor.” The text of the Creative Commons
+public licenses is dedicated to the public domain under the CC0 Public
+Domain Dedication. Except for the limited purpose of indicating that
+material is shared under a Creative Commons public license or as
+otherwise permitted by the Creative Commons policies published at
+creativecommons.org/policies, Creative Commons does not authorize the
+use of the trademark "Creative Commons" or any other trademark or logo
+of Creative Commons without its prior written consent including,
+without limitation, in connection with any unauthorized modifications
+to any of its public licenses or any other arrangements,
+understandings, or agreements concerning use of licensed material. For
+the avoidance of doubt, this paragraph does not form part of the
+public licenses.
+
+Creative Commons may be contacted at creativecommons.org.

Notebooks/Chap01/1_1_BackgroundMathematics.ipynb

@@ -19,7 +19,7 @@
         "\n",
         "# **Notebook 1.1 -- Background Mathematics**\n",
         "\n",
-        "The purpose of this Python notebook is to make sure you can use CoLab and to familiarize yourself with some of the background mathematical concepts that you are going to need to understand deep learning. <br><br> It's not meant to be difficult and it may be that you know some or all of this information already.<br><br> Math is *NOT* a spectator sport.  You won't learn it by just listening to lectures or reading books.  It really helps to interact with it and explore yourself. <br><br> Work through the cells below, running each cell in turn.  In various places you will see the words **\"TO DO\"**. Follow the instructions at these places and write code to complete the functions.  There are also questions interspersed in the text.\n",
+        "The purpose of this Python notebook is to make sure you can use CoLab and to familiarize yourself with some of the background mathematical concepts that you are going to need to understand deep learning. <br><br> It's not meant to be difficult and it may be that you know some or all of this information already.<br><br> Math is *NOT* a spectator sport.  You won't learn it by just listening to lectures or reading books.  It really helps to interact with it and explore yourself. <br><br> Work through the cells below, running each cell in turn.  In various places you will see the words **\"TODO\"**. Follow the instructions at these places and write code to complete the functions.  There are also questions interspersed in the text.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ]
@@ -46,11 +46,11 @@
       "source": [
         "**Linear functions**<br> We will be using the term *linear equation* to mean a weighted sum of inputs plus an offset. If there is just one input $x$, then this is a straight line:\n",
         "\n",
-        "\\begin{equation}y=\\beta+\\omega x,\\end{equation} <br>\n",
+        "\\begin{equation}y=\\beta+\\omega x,\\end{equation}\n",
         "\n",
         "where $\\beta$ is the y-intercept of the linear and $\\omega$ is the slope of the line. When there are two inputs $x_{1}$ and $x_{2}$, then this becomes:\n",
         "\n",
-        "\\begin{equation}y=\\beta+\\omega_1 x_1 + \\omega_2 x_2.\\end{equation} <br><br>\n",
+        "\\begin{equation}y=\\beta+\\omega_1 x_1 + \\omega_2 x_2.\\end{equation}\n",
         "\n",
         "Any other functions are by definition **non-linear**.\n",
         "\n",
@@ -83,7 +83,7 @@
       "source": [
         "# Plot the 1D linear function\n",
         "\n",
-        "# Define an array of x values from 0 to 10 with increments of 0.1\n",
+        "# Define an array of x values from 0 to 10 with increments of 0.01\n",
         "# https://numpy.org/doc/stable/reference/generated/numpy.arange.html\n",
         "x = np.arange(0.0,10.0, 0.01)\n",
         "# Compute y using the function you filled in above\n",
@@ -96,7 +96,7 @@
         "ax.plot(x,y,'r-')\n",
         "ax.set_ylim([0,10]);ax.set_xlim([0,10])\n",
         "ax.set_xlabel('x'); ax.set_ylabel('y')\n",
-        "plt.show\n",
+        "plt.show()\n",
         "\n",
         "# TODO -- experiment with changing the values of beta and omega\n",
         "# to understand what they do.  Try to make a line\n",
@@ -171,7 +171,7 @@
         "# Color represents y value (brighter = higher value)\n",
         "# Black = -10 or less, White = +10 or more\n",
         "# 0 = mid orange\n",
-        "# Lines are conoturs where value is equal\n",
+        "# Lines are contours where value is equal\n",
         "draw_2D_function(x1,x2,y)\n",
         "\n",
         "# TODO\n",
@@ -195,15 +195,15 @@
       "source": [
         "Often we will want to compute many linear functions at the same time.  For example, we might have three inputs, $x_1$, $x_2$, and $x_3$ and want to compute two linear functions giving $y_1$ and $y_2$. Of course, we could do this by just running each equation separately,<br><br>\n",
         "\n",
-        "\\begin{eqnarray}y_1 &=& \\beta_1 + \\omega_{11} x_1 + \\omega_{12} x_2 + \\omega_{13} x_3\\\\\n",
+        "\\begin{align}y_1 &=& \\beta_1 + \\omega_{11} x_1 + \\omega_{12} x_2 + \\omega_{13} x_3\\\\\n",
         "y_2 &=& \\beta_2 + \\omega_{21} x_1 + \\omega_{22} x_2 + \\omega_{23} x_3.\n",
-        "\\end{eqnarray}<br>\n",
+        "\\end{align}\n",
         "\n",
         "However, we can write it more compactly with vectors and matrices:\n",
         "\n",
         "\\begin{equation}\n",
         "\\begin{bmatrix} y_1\\\\ y_2 \\end{bmatrix} = \\begin{bmatrix}\\beta_{1}\\\\\\beta_{2}\\end{bmatrix}+ \\begin{bmatrix}\\omega_{11}&\\omega_{12}&\\omega_{13}\\\\\\omega_{21}&\\omega_{22}&\\omega_{23}\\end{bmatrix}\\begin{bmatrix}x_{1}\\\\x_{2}\\\\x_{3}\\end{bmatrix},\n",
-        "\\end{equation}<br>\n",
+        "\\end{equation}\n",
         "or\n",
         "\n",
         "\\begin{equation}\n",
@@ -269,7 +269,7 @@
         "# Compute with vector/matrix form\n",
         "y_vec = beta_vec+np.matmul(omega_mat, x_vec)\n",
         "print(\"Matrix/vector form\")\n",
-        "print('y1= %3.3f\\ny2 = %3.3f'%((y_vec[0],y_vec[1])))\n"
+        "print('y1= %3.3f\\ny2 = %3.3f'%((y_vec[0][0],y_vec[1][0])))\n"
       ]
     },
     {
@@ -295,7 +295,7 @@
         "\n",
         "Throughout the book, we'll be using some special functions (see Appendix B.1.3).  The most important of these are the logarithm and exponential functions.  Let's investigate their properties.\n",
         "\n",
-        "We'll start with the exponential function $y=\\mbox{exp}[x]=e^x$ which maps the real line $[-\\infty,+\\infty]$ to non-negative numbers $[0,+\\infty]$."
+        "We'll start with the exponential function $y=\\exp[x]=e^x$ which maps the real line $(-\\infty,+\\infty)$ to positive numbers $(0,+\\infty)$."
       ]
     },
     {
@@ -308,7 +308,7 @@
       "source": [
         "# Draw the exponential function\n",
         "\n",
-        "# Define an array of x values from -5 to 5 with increments of 0.1\n",
+        "# Define an array of x values from -5 to 5 with increments of 0.01\n",
         "x = np.arange(-5.0,5.0, 0.01)\n",
         "y = np.exp(x) ;\n",
         "\n",
@@ -317,7 +317,7 @@
         "ax.plot(x,y,'r-')\n",
         "ax.set_ylim([0,100]);ax.set_xlim([-5,5])\n",
         "ax.set_xlabel('x'); ax.set_ylabel('exp[x]')\n",
-        "plt.show"
+        "plt.show()"
       ]
     },
     {
@@ -328,11 +328,11 @@
       "source": [
         "# Questions\n",
         "\n",
-        "1. What is $\\mbox{exp}[0]$?  \n",
-        "2. What is $\\mbox{exp}[1]$?\n",
-        "3. What is $\\mbox{exp}[-\\infty]$?\n",
-        "4. What is $\\mbox{exp}[+\\infty]$?\n",
-        "5. A function is convex if we can draw a straight line between any two points on the function, and this line always lies above the function. Similarly, a function is concave if a straight line between any two points always lies below the function.  Is the exponential function convex or concave or neither?\n"
+        "1. What is $\\exp[0]$?  \n",
+        "2. What is $\\exp[1]$?\n",
+        "3. What is $\\exp[-\\infty]$?\n",
+        "4. What is $\\exp[+\\infty]$?\n",
+        "5. A function is convex if we can draw a straight line between any two points on the function, and the line lies above the function everywhere between these two points. Similarly, a function is concave if a straight line between any two points lies below the function everywhere between these two points.  Is the exponential function convex or concave or neither?\n"
       ]
     },
     {
@@ -354,7 +354,7 @@
       "source": [
         "# Draw the logarithm function\n",
         "\n",
-        "# Define an array of x values from -5 to 5 with increments of 0.1\n",
+        "# Define an array of x values from -5 to 5 with increments of 0.01\n",
         "x = np.arange(0.01,5.0, 0.01)\n",
         "y = np.log(x) ;\n",
         "\n",
@@ -363,7 +363,7 @@
         "ax.plot(x,y,'r-')\n",
         "ax.set_ylim([-5,5]);ax.set_xlim([0,5])\n",
         "ax.set_xlabel('x'); ax.set_ylabel('$\\log[x]$')\n",
-        "plt.show"
+        "plt.show()"
       ]
     },
     {
@@ -374,12 +374,12 @@
       "source": [
         "# Questions\n",
         "\n",
-        "1. What is $\\mbox{log}[0]$?  \n",
-        "2. What is $\\mbox{log}[1]$?\n",
-        "3. What is $\\mbox{log}[e]$?\n",
-        "4. What is $\\mbox{log}[\\exp[3]]$?\n",
-        "5. What is $\\mbox{exp}[\\log[4]]$?\n",
-        "6. What is $\\mbox{log}[-1]$?\n",
+        "1. What is $\\log[0]$?  \n",
+        "2. What is $\\log[1]$?\n",
+        "3. What is $\\log[e]$?\n",
+        "4. What is $\\log[\\exp[3]]$?\n",
+        "5. What is $\\exp[\\log[4]]$?\n",
+        "6. What is $\\log[-1]$?\n",
         "7. Is the logarithm function concave or convex?\n"
       ]
     }
@@ -409,4 +409,4 @@
   },
   "nbformat": 4,
   "nbformat_minor": 0
-}
+}

Notebooks/Chap02/2_1_Supervised_Learning.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyOmndC0N7dFV7W3Mh5ljOLl",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -31,9 +30,9 @@
       "source": [
         "# Notebook 2.1 Supervised Learning\n",
         "\n",
-        "The purpose of this notebook is to explore the linear regression model dicussed in Chapter 2 of the book.\n",
+        "The purpose of this notebook is to explore the linear regression model discussed in Chapter 2 of the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -175,7 +174,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# TO DO -- Change the parameters manually to fit the model\n",
+        "# TODO -- Change the parameters manually to fit the model\n",
         "# First fix phi1 and try changing phi0 until you can't make the loss go down any more\n",
         "# Then fix phi0 and try changing phi1 until you can't make the loss go down any more\n",
         "# Repeat this process until you find a set of parameters that fit the model as in figure 2.2d\n",
@@ -197,7 +196,7 @@
       "source": [
         "# Visualizing the loss function\n",
         "\n",
-        "The above process is equivalent to to descending coordinate wise on the loss function<br>\n",
+        "The above process is equivalent to descending coordinate wise on the loss function<br>\n",
         "\n",
         "Now let's plot that function"
       ],
@@ -213,7 +212,7 @@
         "\n",
         "# Make a 2D array for the losses\n",
         "all_losses = np.zeros_like(phi1_mesh)\n",
-        "# Run throught each 2D combination of phi0, phi1 and compute loss\n",
+        "# Run through each 2D combination of phi0, phi1 and compute loss\n",
         "for indices,temp in np.ndenumerate(phi1_mesh):\n",
         "    all_losses[indices] = compute_loss(x,y, phi0_mesh[indices], phi1_mesh[indices])\n"
       ],
@@ -235,8 +234,8 @@
         "levels = 40\n",
         "ax.contour(phi0_mesh, phi1_mesh, all_losses ,levels, colors=['#80808080'])\n",
         "ax.set_ylim([1,-1])\n",
-        "ax.set_xlabel('Intercept, $\\phi_0$')\n",
-        "ax.set_ylabel('Slope, $\\phi_1$')\n",
+        "ax.set_xlabel(r'Intercept, $\\phi_0$')\n",
+        "ax.set_ylabel(r'Slope, $\\phi_1$')\n",
         "\n",
         "# Plot the position of your best fitting line on the loss function\n",
         "# It should be close to the minimum\n",
@@ -250,4 +249,4 @@
       "outputs": []
     }
   ]
-}
+}

Notebooks/Chap03/3_2_Shallow_Networks_II.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyNk2dAhwwRxGpfVSC3b2Owv",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +32,7 @@
         "\n",
         "The purpose of this notebook is to gain some familiarity with shallow neural networks with 2D inputs.  It works through an example similar to figure 3.8 and experiments with different activation functions. <br><br>\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
       ],
@@ -182,7 +181,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Now we'll extend this model to have two outputs $y_1$ and $y_2$, each of which can be visualized with a separate heatmap.  You will now have sets of parameters $\\phi_{10}, \\phi_{11},\\phi_{12}$ and $\\phi_{20}, \\phi_{21},\\phi_{22}$ that correspond to each of these outputs."
+        "Now we'll extend this model to have two outputs $y_1$ and $y_2$, each of which can be visualized with a separate heatmap.  You will now have sets of parameters $\\phi_{10}, \\phi_{11}, \\phi_{12}, \\phi_{13}$ and $\\phi_{20}, \\phi_{21}, \\phi_{22}, \\phi_{23}$ that correspond to each of these outputs."
       ],
       "metadata": {
         "id": "Xl6LcrUyM7Lh"

Notebooks/Chap03/3_3_Shallow_Network_Regions.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyNioITtfAcfxEfM3UOfQyb9",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +32,7 @@
         "\n",
         "The purpose of this notebook is to compute the maximum possible number of linear regions as seen in figure 3.9 of the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -48,7 +47,7 @@
         "import numpy as np\n",
         "# Imports plotting library\n",
         "import matplotlib.pyplot as plt\n",
-        "# Imports math libray\n",
+        "# Imports math library\n",
         "import math"
       ],
       "metadata": {
@@ -62,7 +61,7 @@
       "source": [
         "The number of regions $N$ created by a shallow neural network with $D_i$ inputs and $D$ hidden units is given by Zaslavsky's formula:\n",
         "\n",
-        "\\begin{equation}N = \\sum_{j=0}^{D_{i}}\\binom{D}{j}=\\sum_{j=0}^{D_{i}} \\frac{D!}{(D-j)!j!} \\end{equation} <br>\n",
+        "\\begin{equation}N = \\sum_{j=0}^{D_{i}}\\binom{D}{j}=\\sum_{j=0}^{D_{i}} \\frac{D!}{(D-j)!j!} \\end{equation} \n",
         "\n"
       ],
       "metadata": {
@@ -79,7 +78,7 @@
       "source": [
         "def number_regions(Di, D):\n",
         "  # TODO -- implement Zaslavsky's formula\n",
-        "  # You can use math.com() https://www.w3schools.com/python/ref_math_comb.asp\n",
+        "  # You can use math.comb() https://www.w3schools.com/python/ref_math_comb.asp\n",
         "  # Replace this code\n",
         "  N = 1;\n",
         "\n",
@@ -102,7 +101,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Calculate the number of regions for 10D input (Di=2) and 50 hidden units (D=50)\n",
+        "# Calculate the number of regions for 10D input (Di=10) and 50 hidden units (D=50)\n",
         "N = number_regions(10, 50)\n",
         "print(f\"Di=10, D=50, Number of regions = {int(N)}, True value = 13432735556\")"
       ],
@@ -126,7 +125,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Show that calculation fails when $D_i < D$\n",
+        "# Depending on how you implemented it, the calculation may fail when $D_i > D$ (not to worry...)\n",
         "try:\n",
         "  N = number_regions(10, 8)\n",
         "  print(f\"Di=10, D=8, Number of regions = {int(N)}, True value = 256\")\n",
@@ -221,7 +220,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Now let's plot the graph from figure 3.9a (takes ~1min)\n",
+        "# Now let's plot the graph from figure 3.9b (takes ~1min)\n",
         "dims = np.array([1,5,10,50,100])\n",
         "regions = np.zeros((dims.shape[0], 200))\n",
         "params = np.zeros((dims.shape[0], 200))\n",
@@ -256,4 +255,4 @@
       "outputs": []
     }
   ]
-}
+}

Notebooks/Chap03/3_4_Activation_Functions.ipynb

@@ -1,45 +1,31 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "authorship_tag": "ABX9TyPmra+JD+dm2M3gCqx3bMak",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
+      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
-        "id": "view-in-github",
-        "colab_type": "text"
+        "colab_type": "text",
+        "id": "view-in-github"
       },
       "source": [
         "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/Notebooks/Chap03/3_4_Activation_Functions.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
       ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
+      "metadata": {
+        "id": "Mn0F56yY8ohX"
+      },
       "source": [
         "# **Notebook 3.4 -- Activation functions**\n",
         "\n",
         "The purpose of this practical is to experiment with different activation functions. <br>\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
-      ],
-      "metadata": {
-        "id": "Mn0F56yY8ohX"
-      }
+      ]
     },
     {
       "cell_type": "code",
@@ -57,6 +43,11 @@
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "AeHzflFt9Tgn"
+      },
+      "outputs": [],
       "source": [
         "# Plot the shallow neural network.  We'll assume input in is range [0,1] and output [-1,1]\n",
         "# If the plot_all flag is set to true, then we'll plot all the intermediate stages as in Figure 3.3\n",
@@ -94,15 +85,15 @@
         "    for i in range(len(x_data)):\n",
         "      ax.plot(x_data[i], y_data[i],)\n",
         "  plt.show()"
-      ],
-      "metadata": {
-        "id": "AeHzflFt9Tgn"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "7qeIUrh19AkH"
+      },
+      "outputs": [],
       "source": [
         "# Define a shallow neural network with, one input, one output, and three hidden units\n",
         "def shallow_1_1_3(x, activation_fn, phi_0,phi_1,phi_2,phi_3, theta_10, theta_11, theta_20, theta_21, theta_30, theta_31):\n",
@@ -123,38 +114,39 @@
         "\n",
         "  # Return everything we have calculated\n",
         "  return y, pre_1, pre_2, pre_3, act_1, act_2, act_3, w_act_1, w_act_2, w_act_3"
-      ],
-      "metadata": {
-        "id": "7qeIUrh19AkH"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "cwTp__Fk9YUx"
+      },
+      "outputs": [],
       "source": [
         "# Define the Rectified Linear Unit (ReLU) function\n",
         "def ReLU(preactivation):\n",
         "  activation = preactivation.clip(0.0)\n",
         "  return activation"
-      ],
-      "metadata": {
-        "id": "cwTp__Fk9YUx"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "First, let's run the network with a ReLU functions"
-      ],
       "metadata": {
         "id": "INQkRzyn9kVC"
-      }
+      },
+      "source": [
+        "First, let's run the network with a ReLU functions"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "jT9QuKou9i0_"
+      },
+      "outputs": [],
       "source": [
         "# Now lets define some parameters and run the neural network\n",
         "theta_10 =  0.3 ; theta_11 = -1.0\n",
@@ -170,15 +162,14 @@
         "    shallow_1_1_3(x, ReLU, phi_0,phi_1,phi_2,phi_3, theta_10, theta_11, theta_20, theta_21, theta_30, theta_31)\n",
         "# And then plot it\n",
         "plot_neural(x, y, pre_1, pre_2, pre_3, act_1, act_2, act_3, w_act_1, w_act_2, w_act_3, plot_all=True)"
-      ],
-      "metadata": {
-        "id": "jT9QuKou9i0_"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
+      "metadata": {
+        "id": "-I8N7r1o9HYf"
+      },
       "source": [
         "# Sigmoid activation function\n",
         "\n",
@@ -189,13 +180,15 @@
         "\\end{equation}\n",
         "\n",
         "(Note that the factor of 10 is not standard -- but it allow us to plot on the same axes as the ReLU examples)"
-      ],
-      "metadata": {
-        "id": "-I8N7r1o9HYf"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "hgkioNyr975Y"
+      },
+      "outputs": [],
       "source": [
         "# Define the sigmoid function\n",
         "def sigmoid(preactivation):\n",
@@ -204,15 +197,15 @@
         "  activation = np.zeros_like(preactivation);\n",
         "\n",
         "  return activation"
-      ],
-      "metadata": {
-        "id": "hgkioNyr975Y"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "94HIXKJH97ve"
+      },
+      "outputs": [],
       "source": [
         "# Make an array of inputs\n",
         "z = np.arange(-1,1,0.01)\n",
@@ -223,25 +216,26 @@
         "ax.plot(z,sig_z,'r-')\n",
         "ax.set_xlim([-1,1]);ax.set_ylim([0,1])\n",
         "ax.set_xlabel('z'); ax.set_ylabel('sig[z]')\n",
-        "plt.show"
-      ],
-      "metadata": {
-        "id": "94HIXKJH97ve"
-      },
-      "execution_count": null,
-      "outputs": []
+        "plt.show()"
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Let's see what happens when we use this activation function in a neural network"
-      ],
       "metadata": {
         "id": "p3zQNXhj-J-o"
-      }
+      },
+      "source": [
+        "Let's see what happens when we use this activation function in a neural network"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "C1dASr9L-GNt"
+      },
+      "outputs": [],
       "source": [
         "theta_10 =  0.3 ; theta_11 = -1.0\n",
         "theta_20 = -1.0  ; theta_21 = 2.0\n",
@@ -256,39 +250,41 @@
         "    shallow_1_1_3(x, sigmoid, phi_0,phi_1,phi_2,phi_3, theta_10, theta_11, theta_20, theta_21, theta_30, theta_31)\n",
         "# And then plot it\n",
         "plot_neural(x, y, pre_1, pre_2, pre_3, act_1, act_2, act_3, w_act_1, w_act_2, w_act_3, plot_all=True)"
-      ],
-      "metadata": {
-        "id": "C1dASr9L-GNt"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "You probably notice that this gives nice smooth curves.  So why don't we use this?  Aha... it's not obvious right now, but we will get to it when we learn to fit models."
-      ],
       "metadata": {
         "id": "Uuam_DewA9fH"
-      }
+      },
+      "source": [
+        "You probably notice that this gives nice smooth curves.  So why don't we use this?  Aha... it's not obvious right now, but we will get to it when we learn to fit models."
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
+      "metadata": {
+        "id": "C9WKkcMUABze"
+      },
       "source": [
         "# Heaviside activation function\n",
         "\n",
         "The Heaviside function is defined as:\n",
         "\n",
         "\\begin{equation}\n",
-        "\\mbox{heaviside}[z] = \\begin{cases} 0 & \\quad z <0 \\\\ 1 & \\quad z\\geq 0\\end{cases}\n",
+        "\\text{heaviside}[z] = \\begin{cases} 0 & \\quad z <0 \\\\ 1 & \\quad z\\geq 0\\end{cases}\n",
         "\\end{equation}"
-      ],
-      "metadata": {
-        "id": "C9WKkcMUABze"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "-1qFkdOL-NPc"
+      },
+      "outputs": [],
       "source": [
         "# Define the heaviside function\n",
         "def heaviside(preactivation):\n",
@@ -299,15 +295,15 @@
         "\n",
         "\n",
         "  return activation"
-      ],
-      "metadata": {
-        "id": "-1qFkdOL-NPc"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "mSPyp7iA-44H"
+      },
+      "outputs": [],
       "source": [
         "# Make an array of inputs\n",
         "z = np.arange(-1,1,0.01)\n",
@@ -318,16 +314,16 @@
         "ax.plot(z,heav_z,'r-')\n",
         "ax.set_xlim([-1,1]);ax.set_ylim([-2,2])\n",
         "ax.set_xlabel('z'); ax.set_ylabel('heaviside[z]')\n",
-        "plt.show"
-      ],
-      "metadata": {
-        "id": "mSPyp7iA-44H"
-      },
-      "execution_count": null,
-      "outputs": []
+        "plt.show()"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "t99K2lSl--Mq"
+      },
+      "outputs": [],
       "source": [
         "theta_10 =  0.3 ; theta_11 = -1.0\n",
         "theta_20 = -1.0  ; theta_21 = 2.0\n",
@@ -342,39 +338,41 @@
         "    shallow_1_1_3(x, heaviside, phi_0,phi_1,phi_2,phi_3, theta_10, theta_11, theta_20, theta_21, theta_30, theta_31)\n",
         "# And then plot it\n",
         "plot_neural(x, y, pre_1, pre_2, pre_3, act_1, act_2, act_3, w_act_1, w_act_2, w_act_3, plot_all=True)"
-      ],
-      "metadata": {
-        "id": "t99K2lSl--Mq"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "This can approximate any function, but the output is discontinuous, and there are also reasons not to use it that we will discover when we learn more about model fitting."
-      ],
       "metadata": {
         "id": "T65MRtM-BCQA"
-      }
+      },
+      "source": [
+        "This can approximate any function, but the output is discontinuous, and there are also reasons not to use it that we will discover when we learn more about model fitting."
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
+      "metadata": {
+        "id": "RkB-XZMLBTaR"
+      },
       "source": [
         "# Linear activation functions\n",
         "\n",
         "Neural networks don't work if the activation function is linear.  For example, consider what would happen if the activation function was:\n",
         "\n",
         "\\begin{equation}\n",
-        "\\mbox{lin}[z] = a + bz\n",
+        "\\text{lin}[z] = a + bz\n",
         "\\end{equation}"
-      ],
-      "metadata": {
-        "id": "RkB-XZMLBTaR"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Q59v3saj_jq1"
+      },
+      "outputs": [],
       "source": [
         "# Define the linear activation function\n",
         "def lin(preactivation):\n",
@@ -384,15 +382,15 @@
         "  activation = a+b * preactivation\n",
         "  # Return\n",
         "  return activation"
-      ],
-      "metadata": {
-        "id": "Q59v3saj_jq1"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "IwodsBr0BkDn"
+      },
+      "outputs": [],
       "source": [
         "# TODO\n",
         "# 1. The linear activation function above just returns the input: (0+1*z) = z\n",
@@ -415,12 +413,23 @@
         "    shallow_1_1_3(x, lin, phi_0,phi_1,phi_2,phi_3, theta_10, theta_11, theta_20, theta_21, theta_30, theta_31)\n",
         "# And then plot it\n",
         "plot_neural(x, y, pre_1, pre_2, pre_3, act_1, act_2, act_3, w_act_1, w_act_2, w_act_3, plot_all=True)"
-      ],
-      "metadata": {
-        "id": "IwodsBr0BkDn"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     }
-  ]
-}
+  ],
+  "metadata": {
+    "colab": {
+      "authorship_tag": "ABX9TyOmxhh3ymYWX+1HdZ91I6zU",
+      "include_colab_link": true,
+      "provenance": []
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}

Notebooks/Chap04/4_1_Composing_Networks.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyPEQEGetZqWnLRNn99Q2aaT",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -29,11 +28,11 @@
     {
       "cell_type": "markdown",
       "source": [
-        "#Notebook 4.1 -- Composing networks\n",
+        "# Notebook 4.1 -- Composing networks\n",
         "\n",
         "The purpose of this notebook is to understand what happens when we feed one neural network into another. It works through an example similar to 4.1 and varies both networks\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions"
       ],
@@ -135,7 +134,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Let's define two networks.  We'll put the prefixes n1_ and n2_ before all the variables to make it clear which network is which.  We'll just consider the inputs and outputs over the range [-1,1].  If you set the \"plot_all\" flat to True,  you can see the details of how they were created."
+        "Let's define two networks.  We'll put the prefixes n1_ and n2_ before all the variables to make it clear which network is which.  We'll just consider the inputs and outputs over the range [-1,1]."
       ],
       "metadata": {
         "id": "LxBJCObC-NTY"
@@ -220,7 +219,7 @@
       "source": [
         "# TODO\n",
         "# Take a piece of paper and draw what you think will happen when we feed the\n",
-        "# output of the first network into the second one now that we have changed it.  Draw the relationship between\n",
+        "# output of the first network into the modified second network.  Draw the relationship between\n",
         "# the input of the first network and the output of the second one."
       ],
       "metadata": {
@@ -261,7 +260,7 @@
       "source": [
         "# TODO\n",
         "# Take a piece of paper and draw what you think will happen when we feed the\n",
-        "# output of the first network now we have changed it into the original second network.  Draw the relationship between\n",
+        "# output of the modified first network into the original second network.  Draw the relationship between\n",
         "# the input of the first network and the output of the second one."
       ],
       "metadata": {
@@ -302,7 +301,7 @@
       "source": [
         "# TODO\n",
         "# Take a piece of paper and draw what you think will happen when we feed the\n",
-        "# output of the first network into the original second network.  Draw the relationship between\n",
+        "# output of the first network into the a copy of itself.  Draw the relationship between\n",
         "# the input of the first network and the output of the second one."
       ],
       "metadata": {
@@ -344,13 +343,13 @@
     {
       "cell_type": "code",
       "source": [
-        "# TO DO\n",
+        "# TODO\n",
         "# How many linear regions would there be if we ran N copies of the first network, feeding the result of the first\n",
         "# into the second, the second into the third and so on, and then passed the result into the original second\n",
         "# network (blue curve above)\n",
         "\n",
         "# Take away conclusion:  with very few parameters, we can make A LOT of linear regions, but\n",
-        "# they depend on one another in complex ways that quickly become to difficult to understand intuitively."
+        "# they depend on one another in complex ways that quickly become too difficult to understand intuitively."
       ],
       "metadata": {
         "id": "HqzePCLOVQK7"
@@ -359,4 +358,4 @@
       "outputs": []
     }
   ]
-}
+}

Notebooks/Chap04/4_2_Clipping_functions.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyPkFrjmRAUf0fxN07RC4xMI",
+      "authorship_tag": "ABX9TyPZzptvvf7OPZai8erQ/0xT",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -29,11 +29,11 @@
     {
       "cell_type": "markdown",
       "source": [
-        "#Notebook 4.2 -- Clipping functions\n",
+        "# Notebook 4.2 -- Clipping functions\n",
         "\n",
         "The purpose of this notebook is to understand how a neural network with two hidden layers build more complicated functions by clipping and recombining the representations at the intermediate hidden variables.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions"
       ],
@@ -127,26 +127,26 @@
         "    fig, ax = plt.subplots(3,3)\n",
         "    fig.set_size_inches(8.5, 8.5)\n",
         "    fig.tight_layout(pad=3.0)\n",
-        "    ax[0,0].plot(x,layer2_pre_1,'r-'); ax[0,0].set_ylabel('$\\psi_{10}+\\psi_{11}h_{1}+\\psi_{12}h_{2}+\\psi_{13}h_3$')\n",
-        "    ax[0,1].plot(x,layer2_pre_2,'b-'); ax[0,1].set_ylabel('$\\psi_{20}+\\psi_{21}h_{1}+\\psi_{22}h_{2}+\\psi_{23}h_3$')\n",
-        "    ax[0,2].plot(x,layer2_pre_3,'g-'); ax[0,2].set_ylabel('$\\psi_{30}+\\psi_{31}h_{1}+\\psi_{32}h_{2}+\\psi_{33}h_3$')\n",
-        "    ax[1,0].plot(x,h1_prime,'r-'); ax[1,0].set_ylabel(\"$h_{1}^{'}$\")\n",
-        "    ax[1,1].plot(x,h2_prime,'b-'); ax[1,1].set_ylabel(\"$h_{2}^{'}$\")\n",
-        "    ax[1,2].plot(x,h3_prime,'g-'); ax[1,2].set_ylabel(\"$h_{3}^{'}$\")\n",
-        "    ax[2,0].plot(x,phi1_h1_prime,'r-'); ax[2,0].set_ylabel(\"$\\phi_1 h_{1}^{'}$\")\n",
-        "    ax[2,1].plot(x,phi2_h2_prime,'b-'); ax[2,1].set_ylabel(\"$\\phi_2 h_{2}^{'}$\")\n",
-        "    ax[2,2].plot(x,phi3_h3_prime,'g-'); ax[2,2].set_ylabel(\"$\\phi_3 h_{3}^{'}$\")\n",
+        "    ax[0,0].plot(x,layer2_pre_1,'r-'); ax[0,0].set_ylabel(r'$\\psi_{10}+\\psi_{11}h_{1}+\\psi_{12}h_{2}+\\psi_{13}h_3$')\n",
+        "    ax[0,1].plot(x,layer2_pre_2,'b-'); ax[0,1].set_ylabel(r'$\\psi_{20}+\\psi_{21}h_{1}+\\psi_{22}h_{2}+\\psi_{23}h_3$')\n",
+        "    ax[0,2].plot(x,layer2_pre_3,'g-'); ax[0,2].set_ylabel(r'$\\psi_{30}+\\psi_{31}h_{1}+\\psi_{32}h_{2}+\\psi_{33}h_3$')\n",
+        "    ax[1,0].plot(x,h1_prime,'r-'); ax[1,0].set_ylabel(r\"$h_{1}^{'}$\")\n",
+        "    ax[1,1].plot(x,h2_prime,'b-'); ax[1,1].set_ylabel(r\"$h_{2}^{'}$\")\n",
+        "    ax[1,2].plot(x,h3_prime,'g-'); ax[1,2].set_ylabel(r\"$h_{3}^{'}$\")\n",
+        "    ax[2,0].plot(x,phi1_h1_prime,'r-'); ax[2,0].set_ylabel(r\"$\\phi_1 h_{1}^{'}$\")\n",
+        "    ax[2,1].plot(x,phi2_h2_prime,'b-'); ax[2,1].set_ylabel(r\"$\\phi_2 h_{2}^{'}$\")\n",
+        "    ax[2,2].plot(x,phi3_h3_prime,'g-'); ax[2,2].set_ylabel(r\"$\\phi_3 h_{3}^{'}$\")\n",
         "\n",
         "    for plot_y in range(3):\n",
         "      for plot_x in range(3):\n",
         "        ax[plot_y,plot_x].set_xlim([0,1]);ax[plot_x,plot_y].set_ylim([-1,1])\n",
         "        ax[plot_y,plot_x].set_aspect(0.5)\n",
-        "      ax[2,plot_y].set_xlabel('Input, $x$');\n",
+        "      ax[2,plot_y].set_xlabel(r'Input, $x$');\n",
         "    plt.show()\n",
         "\n",
         "    fig, ax = plt.subplots()\n",
         "    ax.plot(x,y)\n",
-        "    ax.set_xlabel('Input, $x$'); ax.set_ylabel('Output, $y$')\n",
+        "    ax.set_xlabel(r'Input, $x$'); ax.set_ylabel(r'Output, $y$')\n",
         "    ax.set_xlim([0,1]);ax.set_ylim([-1,1])\n",
         "    ax.set_aspect(0.5)\n",
         "    plt.show()"
@@ -169,7 +169,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Define parameters (note first dimension of theta and phi is padded to make indices match\n",
+        "# Define parameters (note first dimension of theta and psi is padded to make indices match\n",
         "# notation in book)\n",
         "theta = np.zeros([4,2])\n",
         "psi = np.zeros([4,4])\n",
@@ -216,4 +216,4 @@
       }
     }
   ]
-}
+}

Notebooks/Chap04/4_3_Deep_Networks.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyPyaqr0yJlxfIcTpfLSHDrP",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -31,9 +30,9 @@
       "source": [
         "# **Notebook 4.3 Deep neural networks**\n",
         "\n",
-        "This network investigates converting neural networks to matrix form.\n",
+        "This notebook investigates converting neural networks to matrix form.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -101,7 +100,6 @@
       "cell_type": "code",
       "source": [
         "# # Plot the shallow neural network.  We'll assume input in is range [-1,1] and output [-1,1]\n",
-        "# If the plot_all flag is set to true, then we'll plot all the intermediate stages as in Figure 3.3\n",
         "def plot_neural(x, y):\n",
         "  fig, ax = plt.subplots()\n",
         "  ax.plot(x.T,y.T)\n",
@@ -119,7 +117,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Let's define a networks.  We'll just consider the inputs and outputs over the range [-1,1].  If you set the \"plot_all\" flat to True,  you can see the details of how it was created."
+        "Let's define a network.  We'll just consider the inputs and outputs over the range [-1,1]."
       ],
       "metadata": {
         "id": "LxBJCObC-NTY"
@@ -151,7 +149,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Now we'll define the same neural network, but this time, we will  use matrix form.  When you get this right, it will draw the same plot as above."
+        "Now we'll define the same neural network, but this time, we will  use matrix form as in equation 4.15.  When you get this right, it will draw the same plot as above."
       ],
       "metadata": {
         "id": "XCJqo_AjfAra"
@@ -177,8 +175,8 @@
         "n1_in_mat = np.reshape(n1_in,(n_dim_in,n_data))\n",
         "\n",
         "# This runs the network for ALL of the inputs, x at once so we can draw graph\n",
-        "h1 = ReLU(np.matmul(beta_0,np.ones((1,n_data))) + np.matmul(Omega_0,n1_in_mat))\n",
-        "n1_out = np.matmul(beta_1,np.ones((1,n_data))) + np.matmul(Omega_1,h1)\n",
+        "h1 = ReLU(beta_0 + np.matmul(Omega_0,n1_in_mat))\n",
+        "n1_out = beta_1 + np.matmul(Omega_1,h1)\n",
         "\n",
         "# Draw the network and check that it looks the same as the non-matrix case\n",
         "plot_neural(n1_in, n1_out)"
@@ -232,7 +230,7 @@
         "beta_2 = np.zeros((1,1))\n",
         "Omega_2 = np.zeros((1,3))\n",
         "\n",
-        "# TODO Fill in the values of the beta and Omega matrices for with the n1_theta, n1_phi, n2_theta, and n2_phi parameters\n",
+        "# TODO Fill in the values of the beta and Omega matrices for the n1_theta, n1_phi, n2_theta, and n2_phi parameters\n",
         "# that define the composition of the two networks above (see eqn 4.5 for Omega1 and beta1 albeit in different notation)\n",
         "# !!! NOTE THAT MATRICES ARE CONVENTIONALLY INDEXED WITH a_11 IN THE TOP LEFT CORNER, BUT NDARRAYS START AT [0,0] SO EVERYTHING IS OFFSET\n",
         "# To get you started I've filled in a few:\n",
@@ -248,9 +246,9 @@
         "n1_in_mat = np.reshape(n1_in,(n_dim_in,n_data))\n",
         "\n",
         "# This runs the network for ALL of the inputs, x at once so we can draw graph (hence extra np.ones term)\n",
-        "h1 = ReLU(np.matmul(beta_0,np.ones((1,n_data))) + np.matmul(Omega_0,n1_in_mat))\n",
-        "h2 = ReLU(np.matmul(beta_1,np.ones((1,n_data))) + np.matmul(Omega_1,h1))\n",
-        "n1_out = np.matmul(beta_2,np.ones((1,n_data))) + np.matmul(Omega_2,h2)\n",
+        "h1 = ReLU(beta_0 + np.matmul(Omega_0,n1_in_mat))\n",
+        "h2 = ReLU(beta_1 + np.matmul(Omega_1,h1))\n",
+        "n1_out = beta_2 + np.matmul(Omega_2,h2)\n",
         "\n",
         "# Draw the network and check that it looks the same as the non-matrix version\n",
         "plot_neural(n1_in, n1_out)"
@@ -292,10 +290,10 @@
         "\n",
         "\n",
         "# If you set the parameters to the correct sizes, the following code will run\n",
-        "h1 = ReLU(np.matmul(beta_0,np.ones((1,n_data))) + np.matmul(Omega_0,x));\n",
-        "h2 = ReLU(np.matmul(beta_1,np.ones((1,n_data))) + np.matmul(Omega_1,h1));\n",
-        "h3 = ReLU(np.matmul(beta_2,np.ones((1,n_data))) + np.matmul(Omega_2,h2));\n",
-        "y = np.matmul(beta_3,np.ones((1,n_data))) + np.matmul(Omega_3,h3)\n",
+        "h1 = ReLU(beta_0 + np.matmul(Omega_0,x));\n",
+        "h2 = ReLU(beta_1 + np.matmul(Omega_1,h1));\n",
+        "h3 = ReLU(beta_2 + np.matmul(Omega_2,h2));\n",
+        "y = beta_3 + np.matmul(Omega_3,h3)\n",
         "\n",
         "if h1.shape[0] is not D_1 or h1.shape[1] is not n_data:\n",
         "  print(\"h1 is wrong shape\")\n",

Notebooks/Chap05/5_1_Least_Squares_Loss.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyPX88BLalmJTle9GSAZMJcz",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +32,7 @@
         "\n",
         "This notebook investigates the least squares loss and the equivalence of maximum likelihood and minimum negative log likelihood.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -66,7 +65,7 @@
         "  return activation\n",
         "\n",
         "# Define a shallow neural network\n",
-        "def shallow_nn(x, beta_0, omega_0, beta_1, omaga_1):\n",
+        "def shallow_nn(x, beta_0, omega_0, beta_1, omega_1):\n",
         "    # Make sure that input data is (1 x n_data) array\n",
         "    n_data = x.size\n",
         "    x = np.reshape(x,(1,n_data))\n",
@@ -119,7 +118,7 @@
         "  ax.plot(x_model,y_model)\n",
         "  if sigma_model is not None:\n",
         "    ax.fill_between(x_model, y_model-2*sigma_model, y_model+2*sigma_model, color='lightgray')\n",
-        "  ax.set_xlabel('Input, $x$'); ax.set_ylabel('Output, $y$')\n",
+        "  ax.set_xlabel(r'Input, $x$'); ax.set_ylabel(r'Output, $y$')\n",
         "  ax.set_xlim([0,1]);ax.set_ylim([-1,1])\n",
         "  ax.set_aspect(0.5)\n",
         "  if title is not None:\n",
@@ -139,7 +138,7 @@
       "source": [
         "# Univariate regression\n",
         "\n",
-        "We'll investigate a simple univarite regression situation with a single input $x$ and a single output $y$ as pictured in figures 5.4 and 5.5b."
+        "We'll investigate a simple univariate regression situation with a single input $x$ and a single output $y$ as pictured in figures 5.4 and 5.5b."
       ],
       "metadata": {
         "id": "PsgLZwsPxauP"
@@ -186,7 +185,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Return probability under normal distribution for input x\n",
+        "# Return probability under normal distribution\n",
         "def normal_distribution(y, mu, sigma):\n",
         "    # TODO-- write in the equation for the normal distribution\n",
         "    # Equation 5.7 from the notes (you will need np.sqrt() and np.exp(), and math.pi)\n",
@@ -223,7 +222,7 @@
         "gauss_prob = normal_distribution(y_gauss, mu, sigma)\n",
         "fig, ax = plt.subplots()\n",
         "ax.plot(y_gauss, gauss_prob)\n",
-        "ax.set_xlabel('Input, $y$'); ax.set_ylabel('Probability $Pr(y)$')\n",
+        "ax.set_xlabel(r'Input, $y$'); ax.set_ylabel(r'Probability $Pr(y)$')\n",
         "ax.set_xlim([-5,5]);ax.set_ylim([0,1.0])\n",
         "plt.show()\n",
         "\n",
@@ -306,7 +305,8 @@
       "source": [
         "# Return the negative log likelihood of the data under the model\n",
         "def compute_negative_log_likelihood(y_train, mu, sigma):\n",
-        "  # TODO -- compute the likelihood of the data -- don't use the likelihood function above -- compute the negative sum of the log probabilities\n",
+        "  # TODO -- compute the negative log likelihood of the data without using a product\n",
+        "  # In other words, compute minus one times the sum of the log probabilities\n",
         "  # Equation 5.4 in the notes\n",
         "  # You will need np.sum(), np.log()\n",
         "  # Replace the line below\n",
@@ -329,7 +329,7 @@
         "mu_pred = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
         "# Set the standard deviation to something reasonable\n",
         "sigma = 0.2\n",
-        "# Compute the log likelihood\n",
+        "# Compute the negative log likelihood\n",
         "nll = compute_negative_log_likelihood(y_train, mu_pred, sigma)\n",
         "# Let's double check we get the right answer before proceeding\n",
         "print(\"Correct answer = %9.9f, Your answer = %9.9f\"%(11.452419564,nll))"
@@ -352,7 +352,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Return the squared distance between the predicted\n",
+        "# Return the squared distance between the observed data (y_train) and the prediction of the model (y_pred)\n",
         "def compute_sum_of_squares(y_train, y_pred):\n",
         "  # TODO -- compute the sum of squared distances between the training data and the model prediction\n",
         "  # Eqn 5.10 in the notes.  Make sure that you understand this, and ask questions if you don't\n",
@@ -372,9 +372,9 @@
       "source": [
         "# Let's test this again\n",
         "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
-        "# Use our neural network to predict the mean of the Gaussian\n",
-        "y_pred = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
-        "# Compute the log likelihood\n",
+        "# Use our neural network to predict the mean of the Gaussian, which is out best prediction of y\n",
+        "y_pred = mu_pred = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+        "# Compute the sum of squares\n",
         "sum_of_squares = compute_sum_of_squares(y_train, y_pred)\n",
         "# Let's double check we get the right answer before proceeding\n",
         "print(\"Correct answer = %9.9f, Your answer = %9.9f\"%(2.020992572,sum_of_squares))"
@@ -388,7 +388,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Now let's investigate finding the maximum likelihood / minimum log likelihood / least squares solution.  For simplicity, we'll assume that all the parameters are correct except one and look at how the likelihood, log likelihood, and sum of squares change as we manipulate the last parameter.  We'll start with overall y offset, beta_1 (formerly phi_0)"
+        "Now let's investigate finding the maximum likelihood / minimum negative log likelihood / least squares solution.  For simplicity, we'll assume that all the parameters are correct except one and look at how the likelihood, negative log likelihood, and sum of squares change as we manipulate the last parameter.  We'll start with overall y offset, beta_1 (formerly phi_0)"
       ],
       "metadata": {
         "id": "OgcRojvPWh4V"
@@ -431,13 +431,26 @@
     {
       "cell_type": "code",
       "source": [
-        "# Now let's plot the likelihood, negative log likelihood, and least squares as a function the value of the offset beta1\n",
-        "fig, ax = plt.subplots(1,3)\n",
-        "fig.set_size_inches(10.5, 3.5)\n",
-        "fig.tight_layout(pad=3.0)\n",
-        "ax[0].plot(beta_1_vals, likelihoods); ax[0].set_xlabel('beta_1[0]$'); ax[0].set_ylabel('likelihood')\n",
-        "ax[1].plot(beta_1_vals, nlls); ax[1].set_xlabel('beta_1[0]'); ax[1].set_ylabel('negative log likelihood')\n",
-        "ax[2].plot(beta_1_vals, sum_squares); ax[2].set_xlabel('beta_1[0]'); ax[2].set_ylabel('sum of squares')\n",
+        "# Now let's plot the likelihood, negative log likelihood, and least squares as a function of the value of the offset beta1\n",
+        "fig, ax = plt.subplots(1,2)\n",
+        "fig.set_size_inches(10.5, 5.5)\n",
+        "fig.tight_layout(pad=10.0)\n",
+        "likelihood_color = 'tab:red'\n",
+        "nll_color = 'tab:blue'\n",
+        "\n",
+        "ax[0].set_xlabel('beta_1[0]')\n",
+        "ax[0].set_ylabel('likelihood', color = likelihood_color)\n",
+        "ax[0].plot(beta_1_vals, likelihoods, color = likelihood_color)\n",
+        "ax[0].tick_params(axis='y', labelcolor=likelihood_color)\n",
+        "\n",
+        "ax00 = ax[0].twinx()\n",
+        "ax00.plot(beta_1_vals, nlls, color = nll_color)\n",
+        "ax00.set_ylabel('negative log likelihood', color = nll_color)\n",
+        "ax00.tick_params(axis='y', labelcolor = nll_color)\n",
+        "\n",
+        "plt.axvline(x = beta_1_vals[np.argmax(likelihoods)], linestyle='dotted')\n",
+        "\n",
+        "ax[1].plot(beta_1_vals, sum_squares); ax[1].set_xlabel('beta_1[0]'); ax[1].set_ylabel('sum of squares')\n",
         "plt.show()"
       ],
       "metadata": {
@@ -517,13 +530,27 @@
     {
       "cell_type": "code",
       "source": [
-        "# Now let's plot the likelihood, negative log likelihood, and least squares as a function the value of the standard divation sigma\n",
-        "fig, ax = plt.subplots(1,3)\n",
-        "fig.set_size_inches(10.5, 3.5)\n",
-        "fig.tight_layout(pad=3.0)\n",
-        "ax[0].plot(sigma_vals, likelihoods); ax[0].set_xlabel('$\\sigma$'); ax[0].set_ylabel('likelihood')\n",
-        "ax[1].plot(sigma_vals, nlls); ax[1].set_xlabel('$\\sigma$'); ax[1].set_ylabel('negative log likelihood')\n",
-        "ax[2].plot(sigma_vals, sum_squares); ax[2].set_xlabel('$\\sigma$'); ax[2].set_ylabel('sum of squares')\n",
+        "# Now let's plot the likelihood, negative log likelihood, and least squares as a function of the value of the standard deviation sigma\n",
+        "fig, ax = plt.subplots(1,2)\n",
+        "fig.set_size_inches(10.5, 5.5)\n",
+        "fig.tight_layout(pad=10.0)\n",
+        "likelihood_color = 'tab:red'\n",
+        "nll_color = 'tab:blue'\n",
+        "\n",
+        "\n",
+        "ax[0].set_xlabel('sigma')\n",
+        "ax[0].set_ylabel('likelihood', color = likelihood_color)\n",
+        "ax[0].plot(sigma_vals, likelihoods, color = likelihood_color)\n",
+        "ax[0].tick_params(axis='y', labelcolor=likelihood_color)\n",
+        "\n",
+        "ax00 = ax[0].twinx()\n",
+        "ax00.plot(sigma_vals, nlls, color = nll_color)\n",
+        "ax00.set_ylabel('negative log likelihood', color = nll_color)\n",
+        "ax00.tick_params(axis='y', labelcolor = nll_color)\n",
+        "\n",
+        "plt.axvline(x = sigma_vals[np.argmax(likelihoods)], linestyle='dotted')\n",
+        "\n",
+        "ax[1].plot(sigma_vals, sum_squares); ax[1].set_xlabel('sigma'); ax[1].set_ylabel('sum of squares')\n",
         "plt.show()"
       ],
       "metadata": {
@@ -538,8 +565,8 @@
         "# Hopefully, you can see that the maximum of the likelihood fn is at the same position as the minimum negative log likelihood\n",
         "# The least squares solution does not depend on sigma, so it's just flat -- no use here.\n",
         "# Let's check that:\n",
-        "print(\"Maximum likelihood = %3.3f, at beta_1=%3.3f\"%( (likelihoods[np.argmax(likelihoods)],sigma_vals[np.argmax(likelihoods)])))\n",
-        "print(\"Minimum negative log likelihood = %3.3f, at beta_1=%3.3f\"%( (nlls[np.argmin(nlls)],sigma_vals[np.argmin(nlls)])))\n",
+        "print(\"Maximum likelihood = %3.3f, at sigma=%3.3f\"%( (likelihoods[np.argmax(likelihoods)],sigma_vals[np.argmax(likelihoods)])))\n",
+        "print(\"Minimum negative log likelihood = %3.3f, at sigma=%3.3f\"%( (nlls[np.argmin(nlls)],sigma_vals[np.argmin(nlls)])))\n",
         "# Plot the best model\n",
         "sigma= sigma_vals[np.argmin(nlls)]\n",
         "y_model = shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
@@ -554,7 +581,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Obviously, to fit the full neural model we would vary all of the 10 parameters of the network in the $\\boldsymbol\\beta_{0},\\boldsymbol\\omega_{0},\\boldsymbol\\beta_{1},\\boldsymbol\\omega_{1}$ (and maybe $\\sigma$) until we find the combination that have the maximum likelihood / minimum negative log likelihood / least squares.<br><br>\n",
+        "Obviously, to fit the full neural model we would vary all of the 10 parameters of the network in $\\boldsymbol\\beta_{0},\\boldsymbol\\Omega_{0},\\boldsymbol\\beta_{1},\\boldsymbol\\Omega_{1}$ (and maybe $\\sigma$) until we find the combination that have the maximum likelihood / minimum negative log likelihood / least squares.<br><br>\n",
         "\n",
         "Here we just varied one at a time as it is easier to see what is going on.  This is known as **coordinate descent**.\n"
       ],

Notebooks/Chap05/5_2_Binary_Cross_Entropy_Loss.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyOlPP7m+YTLyMPaN0WxRdrb",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +32,7 @@
         "\n",
         "This notebook investigates the binary cross-entropy loss.  It follows from applying the formula in section 5.2 to a loss function based on the Bernoulli distribution.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -66,7 +65,7 @@
         "  return activation\n",
         "\n",
         "# Define a shallow neural network\n",
-        "def shallow_nn(x, beta_0, omega_0, beta_1, omaga_1):\n",
+        "def shallow_nn(x, beta_0, omega_0, beta_1, omega_1):\n",
         "    # Make sure that input data is (1 x n_data) array\n",
         "    n_data = x.size\n",
         "    x = np.reshape(x,(1,n_data))\n",
@@ -120,12 +119,12 @@
         "  fig.set_size_inches(7.0, 3.5)\n",
         "  fig.tight_layout(pad=3.0)\n",
         "  ax[0].plot(x_model,out_model)\n",
-        "  ax[0].set_xlabel('Input, $x$'); ax[0].set_ylabel('Model output')\n",
+        "  ax[0].set_xlabel(r'Input, $x$'); ax[0].set_ylabel(r'Model output')\n",
         "  ax[0].set_xlim([0,1]);ax[0].set_ylim([-4,4])\n",
         "  if title is not None:\n",
         "    ax[0].set_title(title)\n",
         "  ax[1].plot(x_model,lambda_model)\n",
-        "  ax[1].set_xlabel('Input, $x$'); ax[1].set_ylabel('$\\lambda$ or Pr(y=1|x)')\n",
+        "  ax[1].set_xlabel(r'Input, $x$'); ax[1].set_ylabel(r'$\\lambda$ or Pr(y=1|x)')\n",
         "  ax[1].set_xlim([0,1]);ax[1].set_ylim([-0.05,1.05])\n",
         "  if title is not None:\n",
         "    ax[1].set_title(title)\n",
@@ -199,7 +198,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "The left is model output and the right is the model output after the sigmoid has been applied, so it now lies in the range [0,1] and represents the probability, that y=1.  The black dots show the training data.  We'll compute the the likelihood and the negative log likelihood."
+        "The left is model output and the right is the model output after the sigmoid has been applied, so it now lies in the range [0,1] and represents the probability, that y=1.  The black dots show the training data.  We'll compute the likelihood and the negative log likelihood."
       ],
       "metadata": {
         "id": "MvVX6tl9AEXF"
@@ -208,7 +207,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Return probability under Bernoulli distribution for input x\n",
+        "# Return probability under Bernoulli distribution for observed class y\n",
         "def bernoulli_distribution(y, lambda_param):\n",
         "    # TODO-- write in the equation for the Bernoulli distribution\n",
         "    # Equation 5.17 from the notes (you will need np.power)\n",
@@ -269,7 +268,7 @@
       "source": [
         "# Let's test this\n",
         "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
-        "# Use our neural network to predict the mean of the Gaussian\n",
+        "# Use our neural network to predict the Bernoulli parameter lambda\n",
         "model_out = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
         "lambda_train = sigmoid(model_out)\n",
         "# Compute the likelihood\n",
@@ -336,7 +335,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Now let's investigate finding the maximum likelihood / minimum negative log likelihood solution.  For simplicity, we'll assume that all the parameters are fixed except one and look at how the likelihood and log likelihood change as we manipulate the last parameter.  We'll start with overall y_offset, beta_1 (formerly phi_0)"
+        "Now let's investigate finding the maximum likelihood / minimum negative log likelihood solution.  For simplicity, we'll assume that all the parameters are fixed except one and look at how the likelihood and negative log likelihood change as we manipulate the last parameter.  We'll start with overall y_offset, beta_1 (formerly phi_0)"
       ],
       "metadata": {
         "id": "OgcRojvPWh4V"
@@ -359,7 +358,7 @@
         "  # Run the network with new parameters\n",
         "  model_out = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
         "  lambda_train = sigmoid(model_out)\n",
-        "  # Compute and store the three values\n",
+        "  # Compute and store the two values\n",
         "  likelihoods[count] = compute_likelihood(y_train,lambda_train)\n",
         "  nlls[count] = compute_negative_log_likelihood(y_train, lambda_train)\n",
         "  # Draw the model for every 20th parameter setting\n",
@@ -378,12 +377,25 @@
     {
       "cell_type": "code",
       "source": [
-        "# Now let's plot the likelihood, and negative log likelihoods as a function the value of the offset beta1\n",
-        "fig, ax = plt.subplots(1,2)\n",
-        "fig.set_size_inches(10.5, 3.5)\n",
-        "fig.tight_layout(pad=3.0)\n",
-        "ax[0].plot(beta_1_vals, likelihoods); ax[0].set_xlabel('beta_1[0]'); ax[0].set_ylabel('likelihood')\n",
-        "ax[1].plot(beta_1_vals, nlls); ax[1].set_xlabel('beta_1[0]'); ax[1].set_ylabel('negative log likelihood')\n",
+        "# Now let's plot the likelihood and negative log likelihood as a function of the value of the offset beta1\n",
+        "fig, ax = plt.subplots()\n",
+        "fig.tight_layout(pad=5.0)\n",
+        "likelihood_color = 'tab:red'\n",
+        "nll_color = 'tab:blue'\n",
+        "\n",
+        "\n",
+        "ax.set_xlabel('beta_1[0]')\n",
+        "ax.set_ylabel('likelihood', color = likelihood_color)\n",
+        "ax.plot(beta_1_vals, likelihoods, color = likelihood_color)\n",
+        "ax.tick_params(axis='y', labelcolor=likelihood_color)\n",
+        "\n",
+        "ax1 = ax.twinx()\n",
+        "ax1.plot(beta_1_vals, nlls, color = nll_color)\n",
+        "ax1.set_ylabel('negative log likelihood', color = nll_color)\n",
+        "ax1.tick_params(axis='y', labelcolor = nll_color)\n",
+        "\n",
+        "plt.axvline(x = beta_1_vals[np.argmax(likelihoods)], linestyle='dotted')\n",
+        "\n",
         "plt.show()"
       ],
       "metadata": {
@@ -417,7 +429,7 @@
       "source": [
         "They both give the same answer. But you can see from the likelihood above that the likelihood is very small unless the parameters are almost correct.  So in practice, we would work with the negative log likelihood.<br><br>\n",
         "\n",
-        "Again, to fit the full neural model we would vary all of the 10 parameters of the network in the $\\boldsymbol\\beta_{0},\\boldsymbol\\omega_{0},\\boldsymbol\\beta_{1},\\boldsymbol\\omega_{1}$ until we find the combination that have the maximum likelihood / minimum negative log likelihood.<br><br>\n",
+        "Again, to fit the full neural model we would vary all of the 10 parameters of the network in the $\\boldsymbol\\beta_{0},\\boldsymbol\\Omega_{0},\\boldsymbol\\beta_{1},\\boldsymbol\\Omega_{1}$ until we find the combination that have the maximum likelihood / minimum negative log likelihood.<br><br>\n",
         "\n"
       ],
       "metadata": {

Notebooks/Chap05/5_3_Multiclass_Cross_entropy_Loss.ipynb

@@ -1,20 +1,4 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "authorship_tag": "ABX9TyOVTohDBtmCCzSEqLJ4J9R/",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
       "cell_type": "markdown",
@@ -28,18 +12,18 @@
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "jSlFkICHwHQF"
+      },
       "source": [
         "# **Notebook 5.3 Multiclass Cross-Entropy Loss**\n",
         "\n",
         "This notebook investigates the multi-class cross-entropy loss.  It follows from applying the formula in section 5.2 to a loss function based on the Categorical distribution.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
-      ],
-      "metadata": {
-        "id": "jSlFkICHwHQF"
-      }
+      ]
     },
     {
       "cell_type": "code",
@@ -61,6 +45,11 @@
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Fv7SZR3tv7mV"
+      },
+      "outputs": [],
       "source": [
         "# Define the Rectified Linear Unit (ReLU) function\n",
         "def ReLU(preactivation):\n",
@@ -68,7 +57,7 @@
         "  return activation\n",
         "\n",
         "# Define a shallow neural network\n",
-        "def shallow_nn(x, beta_0, omega_0, beta_1, omaga_1):\n",
+        "def shallow_nn(x, beta_0, omega_0, beta_1, omega_1):\n",
         "    # Make sure that input data is (1 x n_data) array\n",
         "    n_data = x.size\n",
         "    x = np.reshape(x,(1,n_data))\n",
@@ -77,15 +66,15 @@
         "    h1 = ReLU(np.matmul(beta_0,np.ones((1,n_data))) + np.matmul(omega_0,x))\n",
         "    model_out = np.matmul(beta_1,np.ones((1,n_data))) + np.matmul(omega_1,h1)\n",
         "    return model_out"
-      ],
-      "metadata": {
-        "id": "Fv7SZR3tv7mV"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "pUT9Ain_HRim"
+      },
+      "outputs": [],
       "source": [
         "# Get parameters for model -- we can call this function to easily reset them\n",
         "def get_parameters():\n",
@@ -103,15 +92,15 @@
         "  omega_1[2,0] = 16.0; omega_1[2,1] = -8.0; omega_1[2,2] =-8\n",
         "\n",
         "  return beta_0, omega_0, beta_1, omega_1"
-      ],
-      "metadata": {
-        "id": "pUT9Ain_HRim"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "NRR67ri_1TzN"
+      },
+      "outputs": [],
       "source": [
         "# Utility function for plotting data\n",
         "def plot_multiclass_classification(x_model, out_model, lambda_model, x_data = None, y_data = None, title= None):\n",
@@ -148,26 +137,26 @@
         "      if y_data[i] ==2:\n",
         "        ax[1].plot(x_data[i],-0.05, 'b.')\n",
         "  plt.show()"
-      ],
-      "metadata": {
-        "id": "NRR67ri_1TzN"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "PsgLZwsPxauP"
+      },
       "source": [
         "# Multiclass classification\n",
         "\n",
         "For multiclass classification, the network must predict the probability of $K$ classes, using $K$ outputs.  However, these probability must be non-negative and sum to one, and the network outputs can take arbitrary values.  Hence, we pass the outputs through a softmax function which maps $K$ arbitrary values to $K$ non-negative values that sum to one."
-      ],
-      "metadata": {
-        "id": "PsgLZwsPxauP"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "uFb8h-9IXnIe"
+      },
+      "outputs": [],
       "source": [
         "# Softmax function that maps a vector of arbitrary values to a vector of values that are positive and sum to one.\n",
         "def softmax(model_out):\n",
@@ -184,15 +173,15 @@
         "  softmax_model_out = np.ones_like(model_out)/ exp_model_out.shape[0]\n",
         "\n",
         "  return softmax_model_out"
-      ],
-      "metadata": {
-        "id": "uFb8h-9IXnIe"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "VWzNOt1swFVd"
+      },
+      "outputs": [],
       "source": [
         "\n",
         "# Let's create some 1D training data\n",
@@ -214,62 +203,61 @@
         "model_out= shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
         "lambda_model = softmax(model_out)\n",
         "plot_multiclass_classification(x_model, model_out, lambda_model, x_train, y_train)\n"
-      ],
-      "metadata": {
-        "id": "VWzNOt1swFVd"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "The left is model output and the right is the model output after the softmax has been applied, so it now lies in the range [0,1] and represents the probability, that y=0 (red), 1 (green) and 2 (blue)   The dots at the bottom show the training data with the same color scheme.  So we want the red curve to be high where there are red dots, the green curve to be high where there are green dots, and the blue curve to be high where there are blue dots  We'll compute the the likelihood and the negative log likelihood."
-      ],
       "metadata": {
         "id": "MvVX6tl9AEXF"
-      }
+      },
+      "source": [
+        "The left is model output and the right is the model output after the softmax has been applied, so it now lies in the range [0,1] and represents the probability, that y=0 (red), 1 (green) and 2 (blue).  The dots at the bottom show the training data with the same color scheme.  So we want the red curve to be high where there are red dots, the green curve to be high where there are green dots, and the blue curve to be high where there are blue dots  We'll compute the likelihood and the negative log likelihood."
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "# Return probability under Categorical distribution for input x\n",
-        "# Just take value from row k of lambda param where y =k,\n",
-        "def categorical_distribution(y, lambda_param):\n",
-        "    return np.array([lambda_param[row, i] for i, row in enumerate (y)])"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "YaLdRlEX0FkU"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "# Return probability under categorical distribution for observed class y\n",
+        "# Just take value from row k of lambda param where y =k,\n",
+        "def categorical_distribution(y, lambda_param):\n",
+        "    return np.array([lambda_param[row, i] for i, row in enumerate (y)])"
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "# Let's double check we get the right answer before proceeding\n",
-        "print(\"Correct answer = %3.3f, Your answer = %3.3f\"%(0.2,categorical_distribution(np.array([[0]]),np.array([[0.2],[0.5],[0.3]]))))\n",
-        "print(\"Correct answer = %3.3f, Your answer = %3.3f\"%(0.5,categorical_distribution(np.array([[1]]),np.array([[0.2],[0.5],[0.3]]))))\n",
-        "print(\"Correct answer = %3.3f, Your answer = %3.3f\"%(0.3,categorical_distribution(np.array([[2]]),np.array([[0.2],[0.5],[0.3]]))))\n",
-        "\n"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "4TSL14dqHHbV"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "# Here are three examples\n",
+        "print(categorical_distribution(np.array([[0]]),np.array([[0.2],[0.5],[0.3]])))\n",
+        "print(categorical_distribution(np.array([[1]]),np.array([[0.2],[0.5],[0.3]])))\n",
+        "print(categorical_distribution(np.array([[2]]),np.array([[0.2],[0.5],[0.3]])))"
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Now let's compute the likelihood using this function"
-      ],
       "metadata": {
         "id": "R5z_0dzQMF35"
-      }
+      },
+      "source": [
+        "Now let's compute the likelihood using this function"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "zpS7o6liCx7f"
+      },
+      "outputs": [],
       "source": [
         "# Return the likelihood of all of the data under the model\n",
         "def compute_likelihood(y_train, lambda_param):\n",
@@ -280,93 +268,93 @@
         "  likelihood = 0\n",
         "\n",
         "  return likelihood"
-      ],
-      "metadata": {
-        "id": "zpS7o6liCx7f"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "1hQxBLoVNlr2"
+      },
+      "outputs": [],
       "source": [
         "# Let's test this\n",
         "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
-        "# Use our neural network to predict the mean of the Gaussian\n",
+        "# Use our neural network to predict the parameters of the categorical distribution\n",
         "model_out = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
         "lambda_train = softmax(model_out)\n",
         "# Compute the likelihood\n",
         "likelihood = compute_likelihood(y_train, lambda_train)\n",
         "# Let's double check we get the right answer before proceeding\n",
         "print(\"Correct answer = %9.9f, Your answer = %9.9f\"%(0.000000041,likelihood))"
-      ],
-      "metadata": {
-        "id": "1hQxBLoVNlr2"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "HzphKgPfOvlk"
+      },
       "source": [
         "You can see that this gives a very small answer, even for this small 1D dataset, and with the model fitting quite well.  This is because it is the product of several probabilities, which are all quite small themselves.\n",
         "This will get out of hand pretty quickly with real datasets -- the likelihood will get so small that we can't represent it with normal finite-precision math\n",
         "\n",
         "This is why we use negative log likelihood"
-      ],
-      "metadata": {
-        "id": "HzphKgPfOvlk"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "dsT0CWiKBmTV"
+      },
+      "outputs": [],
       "source": [
         "# Return the negative log likelihood of the data under the model\n",
         "def compute_negative_log_likelihood(y_train, lambda_param):\n",
-        "  # TODO -- compute the likelihood of the data -- don't use the likelihood function above -- compute the negative sum of the log probabilities\n",
+        "  # TODO -- compute the negative log likelihood of the data -- don't use the likelihood function above -- compute the negative sum of the log probabilities\n",
         "  # You will need np.sum(), np.log()\n",
         "  # Replace the line below\n",
         "  nll = 0\n",
         "\n",
         "  return nll"
-      ],
-      "metadata": {
-        "id": "dsT0CWiKBmTV"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "# Let's test this\n",
-        "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
-        "# Use our neural network to predict the mean of the Gaussian\n",
-        "model_out = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
-        "# Pass the outputs through the softmax function\n",
-        "lambda_train = softmax(model_out)\n",
-        "# Compute the log likelihood\n",
-        "nll = compute_negative_log_likelihood(y_train, lambda_train)\n",
-        "# Let's double check we get the right answer before proceeding\n",
-        "print(\"Correct answer = %9.9f, Your answer = %9.9f\"%(17.015457867,nll))"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "nVxUXg9rQmwI"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "# Let's test this\n",
+        "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+        "# Use our neural network to predict the parameters of the categorical distribution\n",
+        "model_out = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+        "# Pass the outputs through the softmax function\n",
+        "lambda_train = softmax(model_out)\n",
+        "# Compute the negative log likelihood\n",
+        "nll = compute_negative_log_likelihood(y_train, lambda_train)\n",
+        "# Let's double check we get the right answer before proceeding\n",
+        "print(\"Correct answer = %9.9f, Your answer = %9.9f\"%(17.015457867,nll))"
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Now let's investigate finding the maximum likelihood / minimum log likelihood solution.  For simplicity, we'll assume that all the parameters are fixed except one and look at how the likelihood and log likelihood change as we manipulate the last parameter.  We'll start with overall y_offset, beta_1 (formerly phi_0)"
-      ],
       "metadata": {
         "id": "OgcRojvPWh4V"
-      }
+      },
+      "source": [
+        "Now let's investigate finding the maximum likelihood / minimum negative log likelihood solution.  For simplicity, we'll assume that all the parameters are fixed except one and look at how the likelihood and negative log likelihood change as we manipulate the last parameter.  We'll start with overall y_offset, $\\beta_1$ (formerly $\\phi_0$)"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "pFKtDaAeVU4U"
+      },
+      "outputs": [],
       "source": [
         "# Define a range of values for the parameter\n",
         "beta_1_vals = np.arange(-2,6.0,0.1)\n",
@@ -382,7 +370,7 @@
         "  # Run the network with new parameters\n",
         "  model_out = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
         "  lambda_train = softmax(model_out)\n",
-        "  # Compute and store the three values\n",
+        "  # Compute and store the two values\n",
         "  likelihoods[count] = compute_likelihood(y_train,lambda_train)\n",
         "  nlls[count] = compute_negative_log_likelihood(y_train, lambda_train)\n",
         "  # Draw the model for every 20th parameter setting\n",
@@ -391,32 +379,45 @@
         "    model_out = shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
         "    lambda_model = softmax(model_out)\n",
         "    plot_multiclass_classification(x_model, model_out, lambda_model, x_train, y_train, title=\"beta1[0,0]=%3.3f\"%(beta_1[0,0]))\n"
-      ],
-      "metadata": {
-        "id": "pFKtDaAeVU4U"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "# Now let's plot the likelihood, negative log likelihood as a function the value of the offset beta1\n",
-        "fig, ax = plt.subplots(1,2)\n",
-        "fig.set_size_inches(10.5, 3.5)\n",
-        "fig.tight_layout(pad=3.0)\n",
-        "ax[0].plot(beta_1_vals, likelihoods); ax[0].set_xlabel('beta_1[0,0]'); ax[0].set_ylabel('likelihood')\n",
-        "ax[1].plot(beta_1_vals, nlls); ax[1].set_xlabel('beta_1[0,0]'); ax[1].set_ylabel('negative log likelihood')\n",
-        "plt.show()"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "UHXeTa9MagO6"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "# Now let's plot the likelihood and negative log likelihood as a function of the value of the offset beta1\n",
+        "fig, ax = plt.subplots()\n",
+        "fig.tight_layout(pad=5.0)\n",
+        "likelihood_color = 'tab:red'\n",
+        "nll_color = 'tab:blue'\n",
+        "\n",
+        "\n",
+        "ax.set_xlabel('beta_1[0, 0]')\n",
+        "ax.set_ylabel('likelihood', color = likelihood_color)\n",
+        "ax.plot(beta_1_vals, likelihoods, color = likelihood_color)\n",
+        "ax.tick_params(axis='y', labelcolor=likelihood_color)\n",
+        "\n",
+        "ax1 = ax.twinx()\n",
+        "ax1.plot(beta_1_vals, nlls, color = nll_color)\n",
+        "ax1.set_ylabel('negative log likelihood', color = nll_color)\n",
+        "ax1.tick_params(axis='y', labelcolor = nll_color)\n",
+        "\n",
+        "plt.axvline(x = beta_1_vals[np.argmax(likelihoods)], linestyle='dotted')\n",
+        "\n",
+        "plt.show()"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "aDEPhddNdN4u"
+      },
+      "outputs": [],
       "source": [
         "# Hopefully, you can see that the maximum of the likelihood fn is at the same position as the minimum negative log likelihood solution\n",
         "# Let's check that:\n",
@@ -428,24 +429,34 @@
         "model_out = shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
         "lambda_model = softmax(model_out)\n",
         "plot_multiclass_classification(x_model, model_out, lambda_model, x_train, y_train, title=\"beta1[0,0]=%3.3f\"%(beta_1[0,0]))\n"
-      ],
-      "metadata": {
-        "id": "aDEPhddNdN4u"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "771G8N1Vk5A2"
+      },
       "source": [
         "They both give the same answer. But you can see from the likelihood above that the likelihood is very small unless the parameters are almost correct.  So in practice, we would work with the negative log likelihood.<br><br>\n",
         "\n",
-        "Again, to fit the full neural model we would vary all of the 16 parameters of the network in the $\\boldsymbol\\beta_{0},\\boldsymbol\\omega_{0},\\boldsymbol\\beta_{1},\\boldsymbol\\omega_{1}$ until we find the combination that have the maximum likelihood / minimum negative log likelihood.<br><br>\n",
+        "Again, to fit the full neural model we would vary all of the 16 parameters of the network in the $\\boldsymbol\\beta_{0},\\boldsymbol\\Omega_{0},\\boldsymbol\\beta_{1},\\boldsymbol\\Omega_{1}$ until we find the combination that have the maximum likelihood / minimum negative log likelihood.<br><br>\n",
         "\n"
-      ],
-      "metadata": {
-        "id": "771G8N1Vk5A2"
-      }
+      ]
     }
-  ]
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
 }

Notebooks/Chap06/6_1_Line_Search.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyOfxeJ15PMkIi4geDTRCz3c",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +32,7 @@
         "\n",
         "This notebook investigates how to find the minimum of a 1D function using line search as described in Figure 6.10.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
       ],
@@ -67,7 +66,7 @@
         "  fig,ax = plt.subplots()\n",
         "  ax.plot(phi_plot,loss_function(phi_plot),'r-')\n",
         "  ax.set_xlim(0,1); ax.set_ylim(0,1)\n",
-        "  ax.set_xlabel('$\\phi$'); ax.set_ylabel('$L[\\phi]$')\n",
+        "  ax.set_xlabel(r'$\\phi$'); ax.set_ylabel(r'$L[\\phi]$')\n",
         "  if a is not None and b is not None and c is not None and d is not None:\n",
         "      plt.axvspan(a, d, facecolor='k', alpha=0.2)\n",
         "      ax.plot([a,a],[0,1],'b-')\n",
@@ -113,7 +112,7 @@
         "    b = 0.33\n",
         "    c = 0.66\n",
         "    d = 1.0\n",
-        "    n_iter  =0;\n",
+        "    n_iter = 0\n",
         "\n",
         "    # While we haven't found the minimum closely enough\n",
         "    while np.abs(b-c) > thresh and n_iter < max_iter:\n",
@@ -131,23 +130,24 @@
         "\n",
         "        print('Iter %d, a=%3.3f, b=%3.3f, c=%3.3f, d=%3.3f'%(n_iter, a,b,c,d))\n",
         "\n",
-        "        # Rule #1 If point A is less than points B, C, and D then halve values of B, C, and D\n",
+        "        # Rule #1 If the HEIGHT at point A is less than the HEIGHT at points B, C, and D then move them to they are half\n",
+        "        # as far from A as they start\n",
         "        # i.e. bring them closer to the original point\n",
         "        # TODO REPLACE THE BLOCK OF CODE BELOW WITH THIS RULE\n",
         "        if (0):\n",
         "          continue;\n",
         "\n",
         "\n",
-        "        # Rule #2 If point b is less than point c then\n",
-        "        #                     then point d becomes point c, and\n",
+        "        # Rule #2 If the HEIGHT at point b is less than the HEIGHT at point c then\n",
+        "        #                     point d becomes point c, and\n",
         "        #                     point b becomes 1/3 between a and new d\n",
         "        #                     point c becomes 2/3 between a and new d\n",
         "        # TODO REPLACE THE BLOCK OF CODE BELOW WITH THIS RULE\n",
         "        if (0):\n",
         "          continue;\n",
         "\n",
-        "        # Rule #3 If point c is less than point b then\n",
-        "        #                     then point a becomes point b, and\n",
+        "        # Rule #3 If the HEIGHT at point c is less than the HEIGHT at point b then\n",
+        "        #                     point a becomes point b, and\n",
         "        #                     point b becomes 1/3 between new a and d\n",
         "        #                     point c becomes 2/3 between new a and d\n",
         "        # TODO REPLACE THE BLOCK OF CODE BELOW WITH THIS RULE\n",

Notebooks/Chap06/6_2_Gradient_Descent.ipynb

@@ -1,20 +1,4 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "authorship_tag": "ABX9TyM/FIXDTd6tZYs6WRzK00hB",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
       "cell_type": "markdown",
@@ -28,19 +12,19 @@
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "el8l05WQEO46"
+      },
       "source": [
         "# **Notebook 6.2 Gradient descent**\n",
         "\n",
-        "This notebook recreates the gradient descent algorithm as shon in figure 6.1.\n",
+        "This notebook recreates the gradient descent algorithm as shown in figure 6.1.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
         "\n"
-      ],
-      "metadata": {
-        "id": "el8l05WQEO46"
-      }
+      ]
     },
     {
       "cell_type": "code",
@@ -59,34 +43,39 @@
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "4cRkrh9MZ58Z"
+      },
+      "outputs": [],
       "source": [
         "# Let's create our training data 12 pairs {x_i, y_i}\n",
         "# We'll try to fit the straight line model to these data\n",
         "data = np.array([[0.03,0.19,0.34,0.46,0.78,0.81,1.08,1.18,1.39,1.60,1.65,1.90],\n",
         "                 [0.67,0.85,1.05,1.00,1.40,1.50,1.30,1.54,1.55,1.68,1.73,1.60]])"
-      ],
-      "metadata": {
-        "id": "4cRkrh9MZ58Z"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "WQUERmb2erAe"
+      },
+      "outputs": [],
       "source": [
         "# Let's define our model -- just a straight line with intercept phi[0] and slope phi[1]\n",
         "def model(phi,x):\n",
         "  y_pred = phi[0]+phi[1] * x\n",
         "  return y_pred"
-      ],
-      "metadata": {
-        "id": "WQUERmb2erAe"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "qFRe9POHF2le"
+      },
+      "outputs": [],
       "source": [
         "# Draw model\n",
         "def draw_model(data,model,phi,title=None):\n",
@@ -102,39 +91,39 @@
         "  if title is not None:\n",
         "    ax.set_title(title)\n",
         "  plt.show()"
-      ],
-      "metadata": {
-        "id": "qFRe9POHF2le"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "TXx1Tpd1Tl-I"
+      },
+      "outputs": [],
       "source": [
         "# Initialize the parameters to some arbitrary values and draw the model\n",
         "phi = np.zeros((2,1))\n",
         "phi[0] = 0.6      # Intercept\n",
         "phi[1] = -0.2      # Slope\n",
         "draw_model(data,model,phi, \"Initial parameters\")\n"
-      ],
-      "metadata": {
-        "id": "TXx1Tpd1Tl-I"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Now lets create compute the sum of squares loss for the training data"
-      ],
       "metadata": {
         "id": "QU5mdGvpTtEG"
-      }
+      },
+      "source": [
+        "Now let's compute the sum of squares loss for the training data"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "I7dqTY2Gg7CR"
+      },
+      "outputs": [],
       "source": [
         "def compute_loss(data_x, data_y, model, phi):\n",
         "  # TODO -- Write this function -- replace the line below\n",
@@ -145,45 +134,45 @@
         "  loss = 0\n",
         "\n",
         "  return loss"
-      ],
-      "metadata": {
-        "id": "I7dqTY2Gg7CR"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Let's just test that we got that right"
-      ],
       "metadata": {
         "id": "eB5DQvU5hYNx"
-      }
+      },
+      "source": [
+        "Let's just test that we got that right"
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "loss = compute_loss(data[0,:],data[1,:],model,np.array([[0.6],[-0.2]]))\n",
-        "print('Your loss = %3.3f, Correct loss = %3.3f'%(loss, 12.367))"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "Ty05UtEEg9tc"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "loss = compute_loss(data[0,:],data[1,:],model,np.array([[0.6],[-0.2]]))\n",
+        "print('Your loss = %3.3f, Correct loss = %3.3f'%(loss, 12.367))"
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Now let's plot the whole loss function"
-      ],
       "metadata": {
         "id": "F3trnavPiHpH"
-      }
+      },
+      "source": [
+        "Now let's plot the whole loss function"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "K-NTHpAAHlCl"
+      },
+      "outputs": [],
       "source": [
         "def draw_loss_function(compute_loss, data,  model, phi_iters = None):\n",
         "  # Define pretty colormap\n",
@@ -210,39 +199,39 @@
         "  ax.set_ylim([1,-1])\n",
         "  ax.set_xlabel('Intercept $\\phi_{0}$'); ax.set_ylabel('Slope, $\\phi_{1}$')\n",
         "  plt.show()"
-      ],
-      "metadata": {
-        "id": "K-NTHpAAHlCl"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "draw_loss_function(compute_loss, data, model)"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "l8HbvIupnTME"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "draw_loss_function(compute_loss, data, model)"
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "s9Duf05WqqSC"
+      },
       "source": [
         "Now let's compute the gradient vector for a given set of parameters:\n",
         "\n",
         "\\begin{equation}\n",
         "\\frac{\\partial L}{\\partial \\boldsymbol\\phi} = \\begin{bmatrix}\\frac{\\partial L}{\\partial \\phi_0} \\\\\\frac{\\partial L}{\\partial \\phi_1} \\end{bmatrix}.\n",
         "\\end{equation}"
-      ],
-      "metadata": {
-        "id": "s9Duf05WqqSC"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "UpswmkL2qwBT"
+      },
+      "outputs": [],
       "source": [
         "# These are in the lecture slides and notes, but worth trying to calculate them yourself to\n",
         "# check that you get them right.  Write out the expression for the sum of squares loss and take the\n",
@@ -254,31 +243,31 @@
         "\n",
         "    # Return the gradient\n",
         "    return np.array([[dl_dphi0],[dl_dphi1]])"
-      ],
-      "metadata": {
-        "id": "UpswmkL2qwBT"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "RS1nEcYVuEAM"
+      },
       "source": [
         "We can check we got this right using a trick known as **finite differences**.  If we evaluate the function and then change one of the parameters by a very small amount and normalize by that amount, we get an approximation to the gradient, so:\n",
         "\n",
-        "\\begin{eqnarray}\n",
+        "\\begin{align}\n",
         "\\frac{\\partial L}{\\partial \\phi_{0}}&\\approx & \\frac{L[\\phi_0+\\delta, \\phi_1]-L[\\phi_0, \\phi_1]}{\\delta}\\\\\n",
         "\\frac{\\partial L}{\\partial \\phi_{1}}&\\approx & \\frac{L[\\phi_0, \\phi_1+\\delta]-L[\\phi_0, \\phi_1]}{\\delta}\n",
-        "\\end{eqnarray}\n",
+        "\\end{align}\n",
         "\n",
-        "We can't do this when there are many parameters;  for a million parameters, we would have to evaluate the loss function two million times, and usually computing the gradients directly is much more efficient."
-      ],
-      "metadata": {
-        "id": "RS1nEcYVuEAM"
-      }
+        "We can't do this when there are many parameters;  for a million parameters, we would have to evaluate the loss function one million plus one times, and usually computing the gradients directly is much more efficient."
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "QuwAHN7yt-gi"
+      },
+      "outputs": [],
       "source": [
         "# Compute the gradient using your function\n",
         "gradient = compute_gradient(data[0,:],data[1,:], phi)\n",
@@ -291,28 +280,28 @@
         "                    compute_loss(data[0,:],data[1,:],model,phi))/delta\n",
         "print(\"Approx gradients: (%3.3f,%3.3f)\"%(dl_dphi0_est,dl_dphi1_est))\n",
         "# There might be small differences in the last significant figure because finite gradients is an approximation\n"
-      ],
-      "metadata": {
-        "id": "QuwAHN7yt-gi"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Now we are ready to perform gradient descent.  We'll need to use our line search routine from part I, which I've reproduced here plus the helper function loss_function_1D that converts from a 2D problem to a 1D problem"
-      ],
       "metadata": {
         "id": "5EIjMM9Fw2eT"
-      }
+      },
+      "source": [
+        "Now we are ready to perform gradient descent.  We'll need to use our line search routine from notebook 6.1, which I've reproduced here plus the helper function loss_function_1D that maps the search along the negative gradient direction in 2D space to a 1D problem (distance along this direction)"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "XrJ2gQjfw1XP"
+      },
+      "outputs": [],
       "source": [
-        "def loss_function_1D(dist_prop, data, model, phi_start, gradient):\n",
+        "def loss_function_1D(dist_prop, data, model, phi_start, search_direction):\n",
         "  # Return the loss after moving this far\n",
-        "  return compute_loss(data[0,:], data[1,:], model, phi_start+ gradient * dist_prop)\n",
+        "  return compute_loss(data[0,:], data[1,:], model, phi_start - search_direction * dist_prop)\n",
         "\n",
         "def line_search(data, model, phi, gradient, thresh=.00001, max_dist = 0.1, max_iter = 15, verbose=False):\n",
         "    # Initialize four points along the range we are going to search\n",
@@ -320,31 +309,31 @@
         "    b = 0.33 * max_dist\n",
         "    c = 0.66 * max_dist\n",
         "    d = 1.0 * max_dist\n",
-        "    n_iter  =0;\n",
+        "    n_iter = 0\n",
         "\n",
         "    # While we haven't found the minimum closely enough\n",
         "    while np.abs(b-c) > thresh and n_iter < max_iter:\n",
         "        # Increment iteration counter (just to prevent an infinite loop)\n",
         "        n_iter = n_iter+1\n",
         "        # Calculate all four points\n",
-        "        lossa = loss_function_1D(a, data, model, phi,gradient)\n",
-        "        lossb = loss_function_1D(b, data, model, phi,gradient)\n",
-        "        lossc = loss_function_1D(c, data, model, phi,gradient)\n",
-        "        lossd = loss_function_1D(d, data, model, phi,gradient)\n",
+        "        lossa = loss_function_1D(a, data, model, phi, gradient)\n",
+        "        lossb = loss_function_1D(b, data, model, phi, gradient)\n",
+        "        lossc = loss_function_1D(c, data, model, phi, gradient)\n",
+        "        lossd = loss_function_1D(d, data, model, phi, gradient)\n",
         "\n",
         "        if verbose:\n",
         "          print('Iter %d, a=%3.3f, b=%3.3f, c=%3.3f, d=%3.3f'%(n_iter, a,b,c,d))\n",
         "          print('a %f, b%f, c%f, d%f'%(lossa,lossb,lossc,lossd))\n",
         "\n",
-        "        # Rule #1 If point A is less than points B, C, and D then halve points B,C, and D\n",
+        "        # Rule #1 If point A is less than points B, C, and D then halve distance from A to points B,C, and D\n",
         "        if np.argmin((lossa,lossb,lossc,lossd))==0:\n",
-        "          b = b/2\n",
-        "          c = c/2\n",
-        "          d = d/2\n",
+        "          b = a+ (b-a)/2\n",
+        "          c = a+ (c-a)/2\n",
+        "          d = a+ (d-a)/2\n",
         "          continue;\n",
         "\n",
         "        # Rule #2 If point b is less than point c then\n",
-        "        #                     then point d becomes point c, and\n",
+        "        #                     point d becomes point c, and\n",
         "        #                     point b becomes 1/3 between a and new d\n",
         "        #                     point c becomes 2/3 between a and new d\n",
         "        if lossb < lossc:\n",
@@ -354,7 +343,7 @@
         "          continue\n",
         "\n",
         "        # Rule #2 If point c is less than point b then\n",
-        "        #                     then point a becomes point b, and\n",
+        "        #                     point a becomes point b, and\n",
         "        #                     point b becomes 1/3 between new a and d\n",
         "        #                     point c becomes 2/3 between new a and d\n",
         "        a = b\n",
@@ -363,32 +352,32 @@
         "\n",
         "    # Return average of two middle points\n",
         "    return (b+c)/2.0"
-      ],
-      "metadata": {
-        "id": "XrJ2gQjfw1XP"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "def gradient_descent_step(phi, data,  model):\n",
-        "  # TODO -- update Phi with the gradient descent step (equation 6.3)\n",
-        "  # 1. Compute the gradient\n",
-        "  # 2. Find the best step size alpha (use negative gradient as going downhill)\n",
-        "  # 3. Update the parameters phi\n",
-        "\n",
-        "  return phi"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "YVq6rmaWRD2M"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "def gradient_descent_step(phi, data,  model):\n",
+        "  # TODO -- update Phi with the gradient descent step (equation 6.3)\n",
+        "  # 1. Compute the gradient (you wrote this function above)\n",
+        "  # 2. Find the best step size alpha using line search function (above)\n",
+        "  # 3. Update the parameters phi based on the gradient and the step size alpha.\n",
+        "\n",
+        "  return phi"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "tOLd0gtdRLLS"
+      },
+      "outputs": [],
       "source": [
         "# Initialize the parameters and draw the model\n",
         "n_steps = 10\n",
@@ -410,12 +399,22 @@
         "\n",
         "# Draw the trajectory on the loss function\n",
         "draw_loss_function(compute_loss, data, model,phi_all)\n"
-      ],
-      "metadata": {
-        "id": "tOLd0gtdRLLS"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     }
-  ]
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
 }

Notebooks/Chap06/6_3_Stochastic_Gradient_Descent.ipynb

@@ -1,20 +1,4 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "authorship_tag": "ABX9TyNk5FN4qlw3pk8BwDVWw1jN",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
       "cell_type": "markdown",
@@ -28,21 +12,21 @@
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "el8l05WQEO46"
+      },
       "source": [
         "# **Notebook 6.3: Stochastic gradient descent**\n",
         "\n",
         "This notebook investigates gradient descent and stochastic gradient descent and recreates figure 6.5 from the book\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
         "\n",
         "\n",
         "\n"
-      ],
-      "metadata": {
-        "id": "el8l05WQEO46"
-      }
+      ]
     },
     {
       "cell_type": "code",
@@ -61,8 +45,13 @@
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "4cRkrh9MZ58Z"
+      },
+      "outputs": [],
       "source": [
-        "# Let's create our training data 30 pairs {x_i, y_i}\n",
+        "# Let's create our training data of 30 pairs {x_i, y_i}\n",
         "# We'll try to fit the Gabor model to these data\n",
         "data = np.array([[-1.920e+00,-1.422e+01,1.490e+00,-1.940e+00,-2.389e+00,-5.090e+00,\n",
         "                 -8.861e+00,3.578e+00,-6.010e+00,-6.995e+00,3.634e+00,8.743e-01,\n",
@@ -74,15 +63,15 @@
         "                  -2.365e-02,5.098e-01,-2.777e-01,3.367e-01,1.927e-01,-2.222e-01,\n",
         "                  6.352e-02,6.888e-03,3.224e-02,1.091e-02,-5.706e-01,-5.258e-02,\n",
         "                  -3.666e-02,1.709e-01,-4.805e-02,2.008e-01,-1.904e-01,5.952e-01]])"
-      ],
-      "metadata": {
-        "id": "4cRkrh9MZ58Z"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "WQUERmb2erAe"
+      },
+      "outputs": [],
       "source": [
         "# Let's define our model\n",
         "def model(phi,x):\n",
@@ -90,15 +79,15 @@
         "  gauss_component = np.exp(-(phi[0] + 0.06 * phi[1] * x) * (phi[0] + 0.06 * phi[1] * x) / 32)\n",
         "  y_pred= sin_component * gauss_component\n",
         "  return y_pred"
-      ],
-      "metadata": {
-        "id": "WQUERmb2erAe"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "qFRe9POHF2le"
+      },
+      "outputs": [],
       "source": [
         "# Draw model\n",
         "def draw_model(data,model,phi,title=None):\n",
@@ -113,39 +102,39 @@
         "  if title is not None:\n",
         "    ax.set_title(title)\n",
         "  plt.show()"
-      ],
-      "metadata": {
-        "id": "qFRe9POHF2le"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "TXx1Tpd1Tl-I"
+      },
+      "outputs": [],
       "source": [
-        "# Initialize the parmaeters and draw the model\n",
+        "# Initialize the parameters and draw the model\n",
         "phi = np.zeros((2,1))\n",
         "phi[0] =  -5     # Horizontal offset\n",
         "phi[1] =  25     # Frequency\n",
         "draw_model(data,model,phi, \"Initial parameters\")\n"
-      ],
-      "metadata": {
-        "id": "TXx1Tpd1Tl-I"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Now lets create compute the sum of squares loss for the training data"
-      ],
       "metadata": {
         "id": "QU5mdGvpTtEG"
-      }
+      },
+      "source": [
+        "Now let's compute the sum of squares loss for the training data"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "I7dqTY2Gg7CR"
+      },
+      "outputs": [],
       "source": [
         "def compute_loss(data_x, data_y, model, phi):\n",
         "  # TODO -- Write this function -- replace the line below\n",
@@ -155,45 +144,45 @@
         "  loss = 0\n",
         "\n",
         "  return loss"
-      ],
-      "metadata": {
-        "id": "I7dqTY2Gg7CR"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Let's just test that we got that right"
-      ],
       "metadata": {
         "id": "eB5DQvU5hYNx"
-      }
+      },
+      "source": [
+        "Let's just test that we got that right"
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "loss = compute_loss(data[0,:],data[1,:],model,np.array([[0.6],[-0.2]]))\n",
-        "print('Your loss = %3.3f, Correct loss = %3.3f'%(loss, 16.419))"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "Ty05UtEEg9tc"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "loss = compute_loss(data[0,:],data[1,:],model,np.array([[0.6],[-0.2]]))\n",
+        "print('Your loss = %3.3f, Correct loss = %3.3f'%(loss, 16.419))"
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Now let's plot the whole loss function"
-      ],
       "metadata": {
         "id": "F3trnavPiHpH"
-      }
+      },
+      "source": [
+        "Now let's plot the whole loss function"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "K-NTHpAAHlCl"
+      },
+      "outputs": [],
       "source": [
         "def draw_loss_function(compute_loss, data,  model, phi_iters = None):\n",
         "  # Define pretty colormap\n",
@@ -204,7 +193,7 @@
         "  b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
         "  my_colormap = ListedColormap(np.vstack((r,g,b)).transpose()/255.0)\n",
         "\n",
-        "  # Make grid of intercept/slope values to plot\n",
+        "  # Make grid of offset/frequency values to plot\n",
         "  offsets_mesh, freqs_mesh = np.meshgrid(np.arange(-10,10.0,0.1), np.arange(2.5,22.5,0.1))\n",
         "  loss_mesh = np.zeros_like(freqs_mesh)\n",
         "  # Compute loss for every set of parameters\n",
@@ -220,39 +209,39 @@
         "  ax.set_ylim([2.5,22.5])\n",
         "  ax.set_xlabel('Offset $\\phi_{0}$'); ax.set_ylabel('Frequency, $\\phi_{1}$')\n",
         "  plt.show()"
-      ],
-      "metadata": {
-        "id": "K-NTHpAAHlCl"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "draw_loss_function(compute_loss, data, model)"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "l8HbvIupnTME"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "draw_loss_function(compute_loss, data, model)"
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "s9Duf05WqqSC"
+      },
       "source": [
         "Now let's compute the gradient vector for a given set of parameters:\n",
         "\n",
         "\\begin{equation}\n",
         "\\frac{\\partial L}{\\partial \\boldsymbol\\phi} = \\begin{bmatrix}\\frac{\\partial L}{\\partial \\phi_0} \\\\\\frac{\\partial L}{\\partial \\phi_1} \\end{bmatrix}.\n",
         "\\end{equation}"
-      ],
-      "metadata": {
-        "id": "s9Duf05WqqSC"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "UpswmkL2qwBT"
+      },
+      "outputs": [],
       "source": [
         "# These came from writing out the expression for the sum of squares loss and taking the\n",
         "# derivative with respect to phi0 and phi1. It was a lot of hassle to get it right!\n",
@@ -281,31 +270,31 @@
         "    dl_dphi1 = gabor_deriv_phi1(data_x, data_y, phi[0],phi[1])\n",
         "    # Return the gradient\n",
         "    return np.array([[dl_dphi0],[dl_dphi1]])"
-      ],
-      "metadata": {
-        "id": "UpswmkL2qwBT"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "RS1nEcYVuEAM"
+      },
       "source": [
         "We can check we got this right using a trick known as **finite differences**.  If we evaluate the function and then change one of the parameters by a very small amount and normalize by that amount, we get an approximation to the gradient, so:\n",
         "\n",
-        "\\begin{eqnarray}\n",
+        "\\begin{align}\n",
         "\\frac{\\partial L}{\\partial \\phi_{0}}&\\approx & \\frac{L[\\phi_0+\\delta, \\phi_1]-L[\\phi_0, \\phi_1]}{\\delta}\\\\\n",
         "\\frac{\\partial L}{\\partial \\phi_{1}}&\\approx & \\frac{L[\\phi_0, \\phi_1+\\delta]-L[\\phi_0, \\phi_1]}{\\delta}\n",
-        "\\end{eqnarray}\n",
+        "\\end{align}\n",
         "\n",
         "We can't do this when there are many parameters;  for a million parameters, we would have to evaluate the loss function two million times, and usually computing the gradients directly is much more efficient."
-      ],
-      "metadata": {
-        "id": "RS1nEcYVuEAM"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "QuwAHN7yt-gi"
+      },
+      "outputs": [],
       "source": [
         "# Compute the gradient using your function\n",
         "gradient = compute_gradient(data[0,:],data[1,:], phi)\n",
@@ -317,24 +306,24 @@
         "dl_dphi1_est = (compute_loss(data[0,:],data[1,:],model,phi+np.array([[0],[delta]])) - \\\n",
         "                    compute_loss(data[0,:],data[1,:],model,phi))/delta\n",
         "print(\"Approx gradients: (%3.3f,%3.3f)\"%(dl_dphi0_est,dl_dphi1_est))\n"
-      ],
-      "metadata": {
-        "id": "QuwAHN7yt-gi"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Now we are ready to perform gradient descent.  We'll need to use our line search routine from Notebook 6.1, which I've reproduced here plus the helper function loss_function_1D that converts from a 2D problem to a 1D problem"
-      ],
       "metadata": {
         "id": "5EIjMM9Fw2eT"
-      }
+      },
+      "source": [
+        "Now we are ready to perform gradient descent.  We'll need to use our line search routine from Notebook 6.1, which I've reproduced here plus the helper function loss_function_1D that converts from a 2D problem to a 1D problem"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "XrJ2gQjfw1XP"
+      },
+      "outputs": [],
       "source": [
         "def loss_function_1D(dist_prop, data, model, phi_start, gradient):\n",
         "  # Return the loss after moving this far\n",
@@ -346,7 +335,7 @@
         "    b = 0.33 * max_dist\n",
         "    c = 0.66 * max_dist\n",
         "    d = 1.0 * max_dist\n",
-        "    n_iter  =0;\n",
+        "    n_iter = 0\n",
         "\n",
         "    # While we haven't found the minimum closely enough\n",
         "    while np.abs(b-c) > thresh and n_iter < max_iter:\n",
@@ -362,15 +351,15 @@
         "          print('Iter %d, a=%3.3f, b=%3.3f, c=%3.3f, d=%3.3f'%(n_iter, a,b,c,d))\n",
         "          print('a %f, b%f, c%f, d%f'%(lossa,lossb,lossc,lossd))\n",
         "\n",
-        "        # Rule #1 If point A is less than points B, C, and D then halve points B,C, and D\n",
+        "        # Rule #1 If point A is less than points B, C, and D then change B,C,D so they are half their current distance from A\n",
         "        if np.argmin((lossa,lossb,lossc,lossd))==0:\n",
-        "          b = b/2\n",
-        "          c = c/2\n",
-        "          d = d/2\n",
+        "          b = a+ (b-a)/2\n",
+        "          c = a+ (c-a)/2\n",
+        "          d = a+ (d-a)/2\n",
         "          continue;\n",
         "\n",
         "        # Rule #2 If point b is less than point c then\n",
-        "        #                     then point d becomes point c, and\n",
+        "        #                     point d becomes point c, and\n",
         "        #                     point b becomes 1/3 between a and new d\n",
         "        #                     point c becomes 2/3 between a and new d\n",
         "        if lossb < lossc:\n",
@@ -380,7 +369,7 @@
         "          continue\n",
         "\n",
         "        # Rule #2 If point c is less than point b then\n",
-        "        #                     then point a becomes point b, and\n",
+        "        #                     point a becomes point b, and\n",
         "        #                     point b becomes 1/3 between new a and d\n",
         "        #                     point c becomes 2/3 between new a and d\n",
         "        a = b\n",
@@ -389,15 +378,15 @@
         "\n",
         "    # Return average of two middle points\n",
         "    return (b+c)/2.0"
-      ],
-      "metadata": {
-        "id": "XrJ2gQjfw1XP"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "YVq6rmaWRD2M"
+      },
+      "outputs": [],
       "source": [
         "def gradient_descent_step(phi, data,  model):\n",
         "  # Step 1:  Compute the gradient\n",
@@ -406,15 +395,15 @@
         "  alpha = line_search(data, model, phi, gradient*-1, max_dist = 2.0)\n",
         "  phi = phi - alpha * gradient\n",
         "  return phi"
-      ],
-      "metadata": {
-        "id": "YVq6rmaWRD2M"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "tOLd0gtdRLLS"
+      },
+      "outputs": [],
       "source": [
         "# Initialize the parameters\n",
         "n_steps = 21\n",
@@ -435,41 +424,41 @@
         "    draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
         "\n",
         "draw_loss_function(compute_loss, data, model,phi_all)\n"
-      ],
-      "metadata": {
-        "id": "tOLd0gtdRLLS"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "# TODO Experiment with starting the optimization in the previous cell in different places\n",
-        "# and show that it heads to a local minimum if we don't start it in the right valley"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "Oi8ZlH0ptLqA"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "# TODO Experiment with starting the optimization in the previous cell in different places\n",
+        "# and show that it heads to a local minimum if we don't start it in the right valley"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "4l-ueLk-oAxV"
+      },
+      "outputs": [],
       "source": [
         "def gradient_descent_step_fixed_learning_rate(phi, data, alpha):\n",
         "  # TODO -- fill in this routine so that we take a fixed size step of size alpha without using line search\n",
         "\n",
         "  return phi"
-      ],
-      "metadata": {
-        "id": "4l-ueLk-oAxV"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "oi9MX_GRpM41"
+      },
+      "outputs": [],
       "source": [
         "# Initialize the parameters\n",
         "n_steps = 21\n",
@@ -490,47 +479,47 @@
         "    draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
         "\n",
         "draw_loss_function(compute_loss, data, model,phi_all)\n"
-      ],
-      "metadata": {
-        "id": "oi9MX_GRpM41"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "In6sQ5YCpMqn"
+      },
+      "outputs": [],
       "source": [
         "# TODO Experiment with the learning rate, alpha.\n",
         "# What happens if you set it too large?\n",
         "# What happens if you set it too small?"
-      ],
-      "metadata": {
-        "id": "In6sQ5YCpMqn"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "VKTC9-1Gpm3N"
+      },
+      "outputs": [],
       "source": [
         "def stochastic_gradient_descent_step(phi, data, alpha, batch_size):\n",
         "  # TODO -- fill in this routine so that we take a fixed size step of size alpha but only using a subset (batch) of the data\n",
         "  # at each step\n",
         "  # You can use the function np.random.permutation to generate a random permutation of the n_data = data.shape[1] indices\n",
         "  # and then just choose the first n=batch_size of these indices.  Then compute the gradient update\n",
-        "  # from just the data with these indices.   More properly, you should sample with replacement, but this will do for now.\n",
+        "  # from just the data with these indices.   More properly, you should sample without replacement, but this will do for now.\n",
         "\n",
         "\n",
         "  return phi"
-      ],
-      "metadata": {
-        "id": "VKTC9-1Gpm3N"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "469OP_UHskJ4"
+      },
+      "outputs": [],
       "source": [
         "# Set the random number generator so you always get same numbers (disable if you don't want this)\n",
         "np.random.seed(1)\n",
@@ -553,34 +542,44 @@
         "    draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
         "\n",
         "draw_loss_function(compute_loss, data, model,phi_all)"
-      ],
-      "metadata": {
-        "id": "469OP_UHskJ4"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "# TODO -- Experiment with different learning rates, starting points, batch sizes, number of steps.  Get a feel for this."
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "LxE2kTa3s29p"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "# TODO -- Experiment with different learning rates, starting points, batch sizes, number of steps.  Get a feel for this."
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "# TODO -- Add a learning rate schedule.  Reduce the learning rate by a factor of beta every M iterations"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "lw4QPOaQTh5e"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "# TODO -- Add a learning rate schedule.  Reduce the learning rate by a factor of beta every M iterations"
+      ]
     }
-  ]
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
 }

Notebooks/Chap06/6_4_Momentum.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyMLS4qeqBTVHGdg9Sds9jND",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +32,7 @@
         "\n",
         "This notebook investigates the use of momentum as illustrated in figure 6.7 from the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
         "\n",
@@ -62,7 +61,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Let's create our training data 30 pairs {x_i, y_i}\n",
+        "# Let's create our training data of 30 pairs {x_i, y_i}\n",
         "# We'll try to fit the Gabor model to these data\n",
         "data = np.array([[-1.920e+00,-1.422e+01,1.490e+00,-1.940e+00,-2.389e+00,-5.090e+00,\n",
         "                 -8.861e+00,3.578e+00,-6.010e+00,-6.995e+00,3.634e+00,8.743e-01,\n",
@@ -123,7 +122,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Initialize the parmaeters and draw the model\n",
+        "# Initialize the parameters and draw the model\n",
         "phi = np.zeros((2,1))\n",
         "phi[0] =  -5     # Horizontal offset\n",
         "phi[1] =  25     # Frequency\n",
@@ -138,7 +137,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Now lets compute the sum of squares loss for the training data and plot the loss function"
+        "Now let's compute the sum of squares loss for the training data and plot the loss function"
       ],
       "metadata": {
         "id": "QU5mdGvpTtEG"
@@ -161,7 +160,7 @@
         "  b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
         "  my_colormap = ListedColormap(np.vstack((r,g,b)).transpose()/255.0)\n",
         "\n",
-        "  # Make grid of intercept/slope values to plot\n",
+        "  # Make grid of offset/frequency values to plot\n",
         "  offsets_mesh, freqs_mesh = np.meshgrid(np.arange(-10,10.0,0.1), np.arange(2.5,22.5,0.1))\n",
         "  loss_mesh = np.zeros_like(freqs_mesh)\n",
         "  # Compute loss for every set of parameters\n",
@@ -366,7 +365,6 @@
         "\n",
         "  # Update the parameters\n",
         "  phi_all[:,c_step+1:c_step+2] = phi_all[:,c_step:c_step+1] - alpha * momentum\n",
-        "  # Measure loss and draw model every 8th step\n",
         "\n",
         "loss =  compute_loss(data[0,:], data[1,:], model, phi_all[:,c_step+1:c_step+2])\n",
         "draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
@@ -377,6 +375,15 @@
       },
       "execution_count": null,
       "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "Note that for this case, Nesterov momentum does not improve the result."
+      ],
+      "metadata": {
+        "id": "F-As4hS8s2nm"
+      }
     }
   ]
-}
+}

Notebooks/Chap06/6_5_Adam.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyNFsCOnucz1nQt7PBEnKeTV",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +32,7 @@
         "\n",
         "This notebook investigates the Adam algorithm as illustrated in figure 6.9 from the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -109,8 +108,8 @@
         "    ax.contour(phi0mesh, phi1mesh, loss_function, 20, colors=['#80808080'])\n",
         "    ax.plot(opt_path[0,:], opt_path[1,:],'-', color='#a0d9d3ff')\n",
         "    ax.plot(opt_path[0,:], opt_path[1,:],'.', color='#a0d9d3ff',markersize=10)\n",
-        "    ax.set_xlabel(\"$\\phi_{0}$\")\n",
-        "    ax.set_ylabel(\"$\\phi_1}$\")\n",
+        "    ax.set_xlabel(r\"$\\phi_{0}$\")\n",
+        "    ax.set_ylabel(r\"$\\phi_{1}$\")\n",
         "    plt.show()"
       ],
       "metadata": {
@@ -169,7 +168,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Because the function changes much faster in $\\phi_1$ than in $\\phi_0$, there is no great step size to choose.  If we set the step size so that it makes sensible progress in the $\\phi_1$, then it takes many iterations to converge.  If we set the step size tso that we make sensible progress in the $\\phi_{0}$ direction, then the path oscillates in the $\\phi_1$ direction.  \n",
+        "Because the function changes much faster in $\\phi_1$ than in $\\phi_0$, there is no great step size to choose.  If we set the step size so that it makes sensible progress in the $\\phi_1$ direction, then it takes many iterations to converge.  If we set the step size so that we make sensible progress in the $\\phi_0$ direction, then the path oscillates in the $\\phi_1$ direction.  \n",
         "\n",
         "This motivates Adam.  At the core of Adam is the idea that we should just determine which way is downhill along each axis (i.e. left/right for $\\phi_0$ or up/down for $\\phi_1$) and move a fixed distance in that direction."
       ],
@@ -186,11 +185,11 @@
         "    for c_step in range(n_steps):\n",
         "        # Measure the gradient as in equation 6.13 (first line)\n",
         "        m = get_loss_gradient(grad_path[0,c_step], grad_path[1,c_step]);\n",
-        "        # TO DO -- compute the squared gradient as in equation 6.13 (second line)\n",
+        "        # TODO -- compute the squared gradient as in equation 6.13 (second line)\n",
         "        # Replace this line:\n",
         "        v = np.ones_like(grad_path[:,0])\n",
         "\n",
-        "        # TO DO -- apply the update rule (equation 6.14)\n",
+        "        # TODO -- apply the update rule (equation 6.14)\n",
         "        # Replace this line:\n",
         "        grad_path[:,c_step+1] = grad_path[:,c_step]\n",
         "\n",
@@ -222,7 +221,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "This moves towards the minimum at a sensible speed, but we never actually converge -- the solution just bounces back and forth between the last two points.  To make it converge, we add momentum to both the estimates of the gradient and the pointwise squared gradient.  We also modify the statistics by a factor that depends on the time to make sure the progress is now slow to start with."
+        "This moves towards the minimum at a sensible speed, but we never actually converge -- the solution just bounces back and forth between the last two points.  To make it converge, we add momentum to both the estimates of the gradient and the pointwise squared gradient.  We also modify the statistics by a factor that depends on the time to make sure the progress is not slow to start with."
       ],
       "metadata": {
         "id": "_6KoKBJdGGI4"
@@ -248,14 +247,14 @@
         "        # Replace this line:\n",
         "        v = v\n",
         "\n",
-        "        # TODO -- Modify the statistics according to euation 6.16\n",
+        "        # TODO -- Modify the statistics according to equation 6.16\n",
         "        # You will need the function np.power\n",
         "        # Replace these lines\n",
         "        m_tilde = m\n",
         "        v_tilde = v\n",
         "\n",
         "\n",
-        "        # TO DO -- apply the update rule (equation 6.17)\n",
+        "        # TODO -- apply the update rule (equation 6.17)\n",
         "        # Replace this line:\n",
         "        grad_path[:,c_step+1] = grad_path[:,c_step]\n",
         "\n",
@@ -285,4 +284,4 @@
       "outputs": []
     }
   ]
-}
+}

Notebooks/Chap07/7_1_Backpropagation_in_Toy_Model.ipynb

@@ -1,99 +1,87 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "authorship_tag": "ABX9TyP5wHK5E7/el+vxU947K3q8",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
+      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
-        "id": "view-in-github",
-        "colab_type": "text"
+        "colab_type": "text",
+        "id": "view-in-github"
       },
       "source": [
         "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/Notebooks/Chap07/7_1_Backpropagation_in_Toy_Model.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
       ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
+      "metadata": {
+        "id": "pOZ6Djz0dhoy"
+      },
       "source": [
         "# **Notebook 7.1: Backpropagation in Toy Model**\n",
         "\n",
         "This notebook computes the derivatives of the toy function discussed in section 7.3 of the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
-      ],
-      "metadata": {
-        "id": "pOZ6Djz0dhoy"
-      }
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
+      "metadata": {
+        "id": "1DmMo2w63CmT"
+      },
       "source": [
         "We're going to investigate how to take the derivatives of functions where one operation is composed with another, which is composed with a third and so on.  For example, consider the model:\n",
         "\n",
         "\\begin{equation}\n",
-        "     \\mbox{f}[x,\\boldsymbol\\phi] = \\beta_3+\\omega_3\\cdot\\cos\\Bigl[\\beta_2+\\omega_2\\cdot\\exp\\bigl[\\beta_1+\\omega_1\\cdot\\sin[\\beta_0+\\omega_0x]\\bigr]\\Bigr],\n",
+        "     \\text{f}[x,\\boldsymbol\\phi] = \\beta_3+\\omega_3\\cdot\\cos\\Bigl[\\beta_2+\\omega_2\\cdot\\exp\\bigl[\\beta_1+\\omega_1\\cdot\\sin[\\beta_0+\\omega_0x]\\bigr]\\Bigr],\n",
         "\\end{equation}\n",
         "\n",
         "with parameters $\\boldsymbol\\phi=\\{\\beta_0,\\omega_0,\\beta_1,\\omega_1,\\beta_2,\\omega_2,\\beta_3,\\omega_3\\}$.<br>\n",
         "\n",
         "This is a composition of the functions $\\cos[\\bullet],\\exp[\\bullet],\\sin[\\bullet]$.   I chose these just because you probably already know the derivatives of these functions:\n",
         "\n",
-        "\\begin{eqnarray*}\n",
+        "\\begin{align}\n",
         " \\frac{\\partial \\cos[z]}{\\partial z} = -\\sin[z] \\quad\\quad \\frac{\\partial \\exp[z]}{\\partial z} = \\exp[z] \\quad\\quad \\frac{\\partial \\sin[z]}{\\partial z} = \\cos[z].\n",
-        "\\end{eqnarray*}\n",
+        "\\end{align}\n",
         "\n",
         "Suppose that we have a least squares loss function:\n",
         "\n",
         "\\begin{equation*}\n",
-        "\\ell_i = (\\mbox{f}[x_i,\\boldsymbol\\phi]-y_i)^2,\n",
+        "\\ell_i = (\\text{f}[x_i,\\boldsymbol\\phi]-y_i)^2,\n",
         "\\end{equation*}\n",
         "\n",
         "Assume that we know the current values of $\\beta_{0},\\beta_{1},\\beta_{2},\\beta_{3},\\omega_{0},\\omega_{1},\\omega_{2},\\omega_{3}$, $x_i$ and $y_i$. We could obviously calculate $\\ell_i$.   But we also want to know how $\\ell_i$ changes when we make a small change to $\\beta_{0},\\beta_{1},\\beta_{2},\\beta_{3},\\omega_{0},\\omega_{1},\\omega_{2}$, or $\\omega_{3}$.  In other words, we want to compute the eight derivatives:\n",
         "\n",
-        "\\begin{eqnarray*}\n",
-        "\\frac{\\partial \\ell_i}{\\partial \\beta_{0}}, \\quad \\frac{\\partial \\ell_i}{\\partial \\beta_{1}}, \\quad \\frac{\\partial \\ell_i}{\\partial \\beta_{2}}, \\quad \\frac{\\partial \\ell_i }{\\partial \\beta_{3}},  \\quad \\frac{\\partial \\ell_i}{\\partial \\omega_{0}}, \\quad \\frac{\\partial \\ell_i}{\\partial \\omega_{1}}, \\quad \\frac{\\partial \\ell_i}{\\partial \\omega_{2}},  \\quad\\mbox{and} \\quad \\frac{\\partial \\ell_i}{\\partial \\omega_{3}}.\n",
-        "\\end{eqnarray*}"
-      ],
-      "metadata": {
-        "id": "1DmMo2w63CmT"
-      }
+        "\\begin{align}\n",
+        "\\frac{\\partial \\ell_i}{\\partial \\beta_{0}}, \\quad \\frac{\\partial \\ell_i}{\\partial \\beta_{1}}, \\quad \\frac{\\partial \\ell_i}{\\partial \\beta_{2}}, \\quad \\frac{\\partial \\ell_i }{\\partial \\beta_{3}},  \\quad \\frac{\\partial \\ell_i}{\\partial \\omega_{0}}, \\quad \\frac{\\partial \\ell_i}{\\partial \\omega_{1}}, \\quad \\frac{\\partial \\ell_i}{\\partial \\omega_{2}},  \\quad\\text{and} \\quad \\frac{\\partial \\ell_i}{\\partial \\omega_{3}}.\n",
+        "\\end{align}"
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "# import library\n",
-        "import numpy as np"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "RIPaoVN834Lj"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "# import library\n",
+        "import numpy as np"
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Let's first define the original function for $y$ and the likelihood term:"
-      ],
       "metadata": {
         "id": "32-ufWhc3v2c"
-      }
+      },
+      "source": [
+        "Let's first define the original function for $y$ and the loss term:"
+      ]
     },
     {
       "cell_type": "code",
@@ -106,103 +94,135 @@
         "def fn(x, beta0, beta1, beta2, beta3, omega0, omega1, omega2, omega3):\n",
         "  return beta3+omega3 * np.cos(beta2 + omega2 * np.exp(beta1 + omega1 * np.sin(beta0 + omega0 * x)))\n",
         "\n",
-        "def likelihood(x, y, beta0, beta1, beta2, beta3, omega0, omega1, omega2, omega3):\n",
+        "def loss(x, y, beta0, beta1, beta2, beta3, omega0, omega1, omega2, omega3):\n",
         "  diff = fn(x, beta0, beta1, beta2, beta3, omega0, omega1, omega2, omega3) - y\n",
         "  return diff * diff"
       ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Now we'll choose some values for the betas and the omegas and x and compute the output of the function:"
-      ],
       "metadata": {
         "id": "y7tf0ZMt5OXt"
-      }
+      },
+      "source": [
+        "Now we'll choose some values for the betas and the omegas and x and compute the output of the function:"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        },
+        "id": "pwvOcCxr41X_",
+        "outputId": "9541922c-dfc4-4b2e-dfa3-3298812155ce"
+      },
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "l_i=0.139\n"
+          ]
+        }
+      ],
       "source": [
         "beta0 = 1.0; beta1 = 2.0; beta2 = -3.0; beta3 = 0.4\n",
         "omega0 = 0.1; omega1 = -0.4; omega2 = 2.0; omega3 = 3.0\n",
-        "x = 2.3; y =2.0\n",
-        "l_i_func = likelihood(x,y,beta0,beta1,beta2,beta3,omega0,omega1,omega2,omega3)\n",
+        "x = 2.3; y = 2.0\n",
+        "l_i_func = loss(x,y,beta0,beta1,beta2,beta3,omega0,omega1,omega2,omega3)\n",
         "print('l_i=%3.3f'%l_i_func)"
-      ],
-      "metadata": {
-        "id": "pwvOcCxr41X_"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
+      "metadata": {
+        "id": "u5w69NeT64yV"
+      },
       "source": [
         "# Computing derivatives by hand\n",
         "\n",
         "We could compute expressions for the derivatives by hand and write code to compute them directly but some have very complex expressions, even for this relatively simple original equation. For example:\n",
         "\n",
-        "\\begin{eqnarray*}\n",
+        "\\begin{align}\n",
         "\\frac{\\partial \\ell_i}{\\partial \\omega_{0}} &=& -2 \\left( \\beta_3+\\omega_3\\cdot\\cos\\Bigl[\\beta_2+\\omega_2\\cdot\\exp\\bigl[\\beta_1+\\omega_1\\cdot\\sin[\\beta_0+\\omega_0\\cdot x_i]\\bigr]\\Bigr]-y_i\\right)\\nonumber \\\\\n",
         "&&\\hspace{0.5cm}\\cdot \\omega_1\\omega_2\\omega_3\\cdot x_i\\cdot\\cos[\\beta_0+\\omega_0 \\cdot x_i]\\cdot\\exp\\Bigl[\\beta_1 + \\omega_1 \\cdot \\sin[\\beta_0+\\omega_0\\cdot x_i]\\Bigr]\\nonumber\\\\\n",
         "&& \\hspace{1cm}\\cdot \\sin\\biggl[\\beta_2+\\omega_2\\cdot \\exp\\Bigl[\\beta_1 + \\omega_1 \\cdot \\sin[\\beta_0+\\omega_0\\cdot x_i]\\Bigr]\\biggr].\n",
-        "\\end{eqnarray*}"
-      ],
-      "metadata": {
-        "id": "u5w69NeT64yV"
-      }
+        "\\end{align}"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "7t22hALp5zkq"
+      },
+      "outputs": [],
       "source": [
         "dldbeta3_func = 2 * (beta3 +omega3 * np.cos(beta2 + omega2 * np.exp(beta1+omega1 * np.sin(beta0+omega0 * x)))-y)\n",
         "dldomega0_func = -2 *(beta3 +omega3 * np.cos(beta2 + omega2 * np.exp(beta1+omega1 * np.sin(beta0+omega0 * x)))-y) * \\\n",
         "              omega1 * omega2 * omega3 * x * np.cos(beta0 + omega0 * x) * np.exp(beta1 +omega1 * np.sin(beta0 + omega0 * x)) *\\\n",
         "              np.sin(beta2 + omega2 * np.exp(beta1+ omega1* np.sin(beta0+omega0 * x)))"
-      ],
-      "metadata": {
-        "id": "7t22hALp5zkq"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Let's make sure this is correct using finite differences:"
-      ],
       "metadata": {
         "id": "iRh4hnu3-H3n"
-      }
+      },
+      "source": [
+        "Let's make sure this is correct using finite differences:"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        },
+        "id": "1O3XmXMx-HlZ",
+        "outputId": "389ed78e-9d8d-4e8b-9e6b-5f20c21407e8"
+      },
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "dydomega0: Function value = 5.246, Finite difference value = 5.246\n"
+          ]
+        }
+      ],
       "source": [
-        "dldomega0_fd = (likelihood(x,y,beta0,beta1,beta2,beta3,omega0+0.00001,omega1,omega2,omega3)-likelihood(x,y,beta0,beta1,beta2,beta3,omega0,omega1,omega2,omega3))/0.00001\n",
+        "dldomega0_fd = (loss(x,y,beta0,beta1,beta2,beta3,omega0+0.00001,omega1,omega2,omega3)-loss(x,y,beta0,beta1,beta2,beta3,omega0,omega1,omega2,omega3))/0.00001\n",
         "\n",
         "print('dydomega0: Function value = %3.3f, Finite difference value = %3.3f'%(dldomega0_func,dldomega0_fd))"
-      ],
-      "metadata": {
-        "id": "1O3XmXMx-HlZ"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "The code to calculate $\\partial l_i/ \\partial \\omega_0$ is a bit of a nightmare.  It's easy to make mistakes, and you can see that some parts of it are repeated (for example, the $\\sin[\\bullet]$ term), which suggests some kind of redundancy in the calculations.  The goal of this practical is to compute the derivatives in a much simpler way.  There will be three steps:"
-      ],
       "metadata": {
         "id": "wS4IPjZAKWTN"
-      }
+      },
+      "source": [
+        "The code to calculate $\\partial l_i/ \\partial \\omega_0$ is a bit of a nightmare.  It's easy to make mistakes, and you can see that some parts of it are repeated (for example, the $\\sin[\\bullet]$ term), which suggests some kind of redundancy in the calculations.  The goal of this practical is to compute the derivatives in a much simpler way.  There will be three steps:"
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
+      "metadata": {
+        "id": "8UWhvDeNDudz"
+      },
       "source": [
         "**Step 1:** Write the original equations as a series of intermediate calculations.\n",
         "\n",
-        "\\begin{eqnarray}\n",
+        "\\begin{align}\n",
         "f_{0} &=& \\beta_{0} + \\omega_{0} x_i\\nonumber\\\\\n",
         "h_{1} &=& \\sin[f_{0}]\\nonumber\\\\\n",
         "f_{1} &=& \\beta_{1} + \\omega_{1}h_{1}\\nonumber\\\\\n",
@@ -211,16 +231,18 @@
         "h_{3} &=& \\cos[f_{2}]\\nonumber\\\\\n",
         "f_{3} &=& \\beta_{3} + \\omega_{3}h_{3}\\nonumber\\\\\n",
         "l_i &=& (f_3-y_i)^2\n",
-        "\\end{eqnarray}\n",
+        "\\end{align}\n",
         "\n",
         "and compute and store the values of all of these intermediate values.  We'll need them to compute the derivatives.<br>  This is called the **forward pass**."
-      ],
-      "metadata": {
-        "id": "8UWhvDeNDudz"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ZWKAq6HC90qV"
+      },
+      "outputs": [],
       "source": [
         "# TODO compute all the f_k and h_k terms\n",
         "# Replace the code below\n",
@@ -233,15 +255,34 @@
         "h3 = 0\n",
         "f3 = 0\n",
         "l_i = 0\n"
-      ],
-      "metadata": {
-        "id": "ZWKAq6HC90qV"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        },
+        "id": "ibxXw7TUW4Sx",
+        "outputId": "4575e3eb-2b16-4e0b-c84e-9c22b443c3ce"
+      },
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "f0: true value = 1.230, your value = 0.000\n",
+            "h1: true value = 0.942, your value = 0.000\n",
+            "f1: true value = 1.623, your value = 0.000\n",
+            "h2: true value = 5.068, your value = 0.000\n",
+            "f2: true value = 7.137, your value = 0.000\n",
+            "h3: true value = 0.657, your value = 0.000\n",
+            "f3: true value = 2.372, your value = 0.000\n",
+            "l_i original = 0.139, l_i from forward pass = 0.000\n"
+          ]
+        }
+      ],
       "source": [
         "# Let's check we got that right:\n",
         "print(\"f0: true value = %3.3f, your value = %3.3f\"%(1.230, f0))\n",
@@ -251,23 +292,22 @@
         "print(\"f2: true value = %3.3f, your value = %3.3f\"%(7.137, f2))\n",
         "print(\"h3: true value = %3.3f, your value = %3.3f\"%(0.657, h3))\n",
         "print(\"f3: true value = %3.3f, your value = %3.3f\"%(2.372, f3))\n",
-        "print(\"like original = %3.3f, like from forward pass = %3.3f\"%(l_i_func, l_i))\n"
-      ],
-      "metadata": {
-        "id": "ibxXw7TUW4Sx"
-      },
-      "execution_count": null,
-      "outputs": []
+        "print(\"l_i original = %3.3f, l_i from forward pass = %3.3f\"%(l_i_func, l_i))\n"
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
+      "metadata": {
+        "id": "jay8NYWdFHuZ"
+      },
       "source": [
-        "**Step 2:** Compute the derivatives of $y$ with respect to the intermediate quantities that we just calculated, but in reverse order:\n",
+        "**Step 2:** Compute the derivatives of $\\ell_i$ with respect to the intermediate quantities that we just calculated, but in reverse order:\n",
         "\n",
-        "\\begin{eqnarray}\n",
+        "\\begin{align}\n",
         "\\quad \\frac{\\partial \\ell_i}{\\partial f_3}, \\quad \\frac{\\partial \\ell_i}{\\partial h_3}, \\quad \\frac{\\partial \\ell_i}{\\partial f_2}, \\quad\n",
-        "\\frac{\\partial \\ell_i}{\\partial h_2}, \\quad \\frac{\\partial \\ell_i}{\\partial f_1}, \\quad \\frac{\\partial \\ell_i}{\\partial h_1},  \\quad\\mbox{and} \\quad \\frac{\\partial \\ell_i}{\\partial f_0}.\n",
-        "\\end{eqnarray}\n",
+        "\\frac{\\partial \\ell_i}{\\partial h_2}, \\quad \\frac{\\partial \\ell_i}{\\partial f_1}, \\quad \\frac{\\partial \\ell_i}{\\partial h_1},  \\quad\\text{and} \\quad \\frac{\\partial \\ell_i}{\\partial f_0}.\n",
+        "\\end{align}\n",
         "\n",
         "The first of these derivatives is straightforward:\n",
         "\n",
@@ -281,11 +321,11 @@
         "\\frac{\\partial \\ell_i}{\\partial h_{3}} =\\frac{\\partial f_{3}}{\\partial h_{3}} \\frac{\\partial \\ell_i}{\\partial f_{3}} .\n",
         "\\end{equation}\n",
         "\n",
-        "The left-hand side asks how $\\ell_i$ changes when $h_{3}$ changes.  The right-hand side says we can decompose this into (i) how $ell_i$ changes when $f_{3}$ changes and how $f_{3}$ changes when $h_{3}$ changes.  So you get a chain of events happening:  $h_{3}$ changes $f_{3}$, which changes $\\ell_i$, and the derivatives represent the effects of this chain.  Notice that we computed the first of these derivatives already and is  $2 (f_3-y)$. We calculated $f_{3}$ in step 1.  The second term is the derivative of $\\beta_{3} + \\omega_{3}h_{3}$ with respect to $h_3$ which is simply $\\omega_3$.  \n",
+        "The left-hand side asks how $\\ell_i$ changes when $h_{3}$ changes.  The right-hand side says we can decompose this into (i) how $\\ell_i$ changes when $f_{3}$ changes and how $f_{3}$ changes when $h_{3}$ changes.  So you get a chain of events happening:  $h_{3}$ changes $f_{3}$, which changes $\\ell_i$, and the derivatives represent the effects of this chain.  Notice that we computed the first of these derivatives already and is  $2 (f_3-y)$. We calculated $f_{3}$ in step 1.  The second term is the derivative of $\\beta_{3} + \\omega_{3}h_{3}$ with respect to $h_3$ which is simply $\\omega_3$.  \n",
         "\n",
         "We can continue in this way, computing the derivatives of the output with respect to these intermediate quantities:\n",
         "\n",
-        "\\begin{eqnarray}\n",
+        "\\begin{align}\n",
         "\\frac{\\partial \\ell_i}{\\partial f_{2}} &=& \\frac{\\partial h_{3}}{\\partial f_{2}}\\left(\n",
         "\\frac{\\partial f_{3}}{\\partial h_{3}}\\frac{\\partial \\ell_i}{\\partial f_{3}} \\right)\n",
         "\\nonumber \\\\\n",
@@ -293,16 +333,18 @@
         "\\frac{\\partial \\ell_i}{\\partial f_{1}} &=& \\frac{\\partial h_{2}}{\\partial f_{1}}\\left( \\frac{\\partial f_{2}}{\\partial h_{2}}\\frac{\\partial h_{3}}{\\partial f_{2}}\\frac{\\partial f_{3}}{\\partial h_{3}}\\frac{\\partial \\ell_i}{\\partial f_{3}} \\right)\\nonumber \\\\\n",
         "\\frac{\\partial \\ell_i}{\\partial h_{1}} &=& \\frac{\\partial f_{1}}{\\partial h_{1}}\\left(\\frac{\\partial h_{2}}{\\partial f_{1}} \\frac{\\partial f_{2}}{\\partial h_{2}}\\frac{\\partial h_{3}}{\\partial f_{2}}\\frac{\\partial f_{3}}{\\partial h_{3}}\\frac{\\partial \\ell_i}{\\partial f_{3}} \\right)\\nonumber \\\\\n",
         "\\frac{\\partial \\ell_i}{\\partial f_{0}} &=& \\frac{\\partial h_{1}}{\\partial f_{0}}\\left(\\frac{\\partial f_{1}}{\\partial h_{1}}\\frac{\\partial h_{2}}{\\partial f_{1}} \\frac{\\partial f_{2}}{\\partial h_{2}}\\frac{\\partial h_{3}}{\\partial f_{2}}\\frac{\\partial f_{3}}{\\partial h_{3}}\\frac{\\partial \\ell_i}{\\partial f_{3}} \\right).\n",
-        "\\end{eqnarray}\n",
+        "\\end{align}\n",
         "\n",
         "In each case, we have already computed all of the terms except the last one in the previous step, and the last term is simple to evaluate.  This is called the **backward pass**."
-      ],
-      "metadata": {
-        "id": "jay8NYWdFHuZ"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "gCQJeI--Egdl"
+      },
+      "outputs": [],
       "source": [
         "# TODO -- Compute the derivatives of the output with respect\n",
         "# to the intermediate computations h_k and f_k (i.e, run the backward pass)\n",
@@ -315,15 +357,33 @@
         "dldf1 = 1\n",
         "dldh1 = 1\n",
         "dldf0 = 1\n"
-      ],
-      "metadata": {
-        "id": "gCQJeI--Egdl"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        },
+        "id": "dS1OrLtlaFr7",
+        "outputId": "414f0862-ae36-4a0e-b68f-4758835b0e23"
+      },
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "dldf3: true value = 0.745, your value = -4.000\n",
+            "dldh3: true value = 2.234, your value = -12.000\n",
+            "dldf2: true value = -1.683, your value = 1.000\n",
+            "dldh2: true value = -3.366, your value = 1.000\n",
+            "dldf1: true value = -17.060, your value = 1.000\n",
+            "dldh1: true value = 6.824, your value = 1.000\n",
+            "dldf0: true value = 2.281, your value = 1.000\n"
+          ]
+        }
+      ],
       "source": [
         "# Let's check we got that right\n",
         "print(\"dldf3: true value = %3.3f, your value = %3.3f\"%(0.745, dldf3))\n",
@@ -333,38 +393,15 @@
         "print(\"dldf1: true value = %3.3f, your value = %3.3f\"%(-17.060, dldf1))\n",
         "print(\"dldh1: true value = %3.3f, your value = %3.3f\"%(6.824, dldh1))\n",
         "print(\"dldf0: true value = %3.3f, your value = %3.3f\"%(2.281, dldf0))"
-      ],
-      "metadata": {
-        "id": "dS1OrLtlaFr7"
-      },
-      "execution_count": null,
-      "outputs": []
-    },
-    {
-      "cell_type": "markdown",
-      "source": [
-        "**Step 3:**  Finally, we consider how the loss~$\\ell_{i}$ changes when we change the parameters $\\beta_{\\bullet}$ and $\\omega_{\\bullet}$. Once more, we apply the chain rule:\n",
-        "\n",
-        "\n",
-        "\n",
-        "\n",
-        "\\begin{eqnarray}\n",
-        "\\frac{\\partial \\ell_i}{\\partial \\beta_{k}} &=& \\frac{\\partial f_{k}}{\\partial \\beta_{k}}\\frac{\\partial \\ell_i}{\\partial f_{k}}\\nonumber \\\\\n",
-        "\\frac{\\partial \\ell_i}{\\partial \\omega_{k}} &=& \\frac{\\partial f_{k}}{\\partial \\omega_{k}}\\frac{\\partial \\ell_i}{\\partial f_{k}}.\n",
-        "\\end{eqnarray}\n",
-        "\n",
-        "\\noindent In each case, the second term on the right-hand side was computed in step 2. When $k>0$, we have~$f_{k}=\\beta_{k}+\\omega_k \\cdot h_{k}$, so:\n",
-        "\n",
-        "\\begin{eqnarray}\n",
-        "\\frac{\\partial f_{k}}{\\partial \\beta_{k}} = 1 \\quad\\quad\\mbox{and}\\quad \\quad \\frac{\\partial f_{k}}{\\partial \\omega_{k}} &=& h_{k}.\n",
-        "\\end{eqnarray}"
-      ],
-      "metadata": {
-        "id": "FlzlThQPGpkU"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "1I2BhqZhGMK6"
+      },
+      "outputs": [],
       "source": [
         "# TODO -- Calculate the final derivatives with respect to the beta and omega terms\n",
         "\n",
@@ -376,15 +413,34 @@
         "dldomega1 = 1\n",
         "dldbeta0 = 1\n",
         "dldomega0 = 1\n"
-      ],
-      "metadata": {
-        "id": "1I2BhqZhGMK6"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        },
+        "id": "38eiOn2aHgHI",
+        "outputId": "1a67a636-e832-471e-e771-54824363158a"
+      },
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "dldbeta3: Your value = 1.000, True value = 0.745\n",
+            "dldomega3: Your value = 1.000, True value = 0.489\n",
+            "dldbeta2: Your value = 1.000, True value = -1.683\n",
+            "dldomega2: Your value = 1.000, True value = -8.530\n",
+            "dldbeta1: Your value = 1.000, True value = -17.060\n",
+            "dldomega1: Your value = 1.000, True value = -16.079\n",
+            "dldbeta0: Your value = 1.000, True value = 2.281\n",
+            "dldomega0: Your value = 1.000, Function value = 5.246, Finite difference value = 5.246\n"
+          ]
+        }
+      ],
       "source": [
         "# Let's check we got them right\n",
         "print('dldbeta3: Your value = %3.3f, True value = %3.3f'%(dldbeta3, 0.745))\n",
@@ -395,21 +451,33 @@
         "print('dldomega1: Your value = %3.3f, True value = %3.3f'%(dldomega1, -16.079))\n",
         "print('dldbeta0: Your value = %3.3f, True value = %3.3f'%(dldbeta0, 2.281))\n",
         "print('dldomega0: Your value = %3.3f, Function value = %3.3f, Finite difference value = %3.3f'%(dldomega0, dldomega0_func, dldomega0_fd))"
-      ],
-      "metadata": {
-        "id": "38eiOn2aHgHI"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Using this method, we can compute the derivatives quite easily without needing to compute very complicated expressions.  In the next practical, we'll apply this same method to a deep neural network."
-      ],
       "metadata": {
         "id": "N2ZhrR-2fNa1"
-      }
+      },
+      "source": [
+        "Using this method, we can compute the derivatives quite easily without needing to compute very complicated expressions.  In the next practical, we'll apply this same method to a deep neural network."
+      ]
     }
-  ]
-}
+  ],
+  "metadata": {
+    "colab": {
+      "authorship_tag": "ABX9TyN7JeDgslwtZcwRCOuGuPFt",
+      "include_colab_link": true,
+      "provenance": []
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}

Notebooks/Chap07/7_2_Backpropagation.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyN2nPVR0imZntgj4Oasyvmo",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +32,7 @@
         "\n",
         "This notebook runs the backpropagation algorithm on a deep neural network as described in section 7.4 of the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -68,7 +67,7 @@
         "# Set seed so we always get the same random numbers\n",
         "np.random.seed(0)\n",
         "\n",
-        "# Number of layers\n",
+        "# Number of hidden layers\n",
         "K = 5\n",
         "# Number of neurons per layer\n",
         "D = 6\n",
@@ -115,9 +114,9 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Now let's run our random network.  The weight matrices $\\boldsymbol\\Omega_{1\\ldots K}$ are the entries of the list \"all_weights\" and the biases $\\boldsymbol\\beta_{1\\ldots k}$ are the entries of the list \"all_biases\"\n",
+        "Now let's run our random network.  The weight matrices $\\boldsymbol\\Omega_{0\\ldots K}$ are the entries of the list \"all_weights\" and the biases $\\boldsymbol\\beta_{0\\ldots K}$ are the entries of the list \"all_biases\"\n",
         "\n",
-        "We know that we will need the activations $\\mathbf{f}_{0\\ldots K}$ and the activations $\\mathbf{h}_{1\\ldots K}$ for the forward pass of backpropagation, so we'll store and return these as well.\n"
+        "We know that we will need the preactivations $\\mathbf{f}_{0\\ldots K}$ and the activations $\\mathbf{h}_{1\\ldots K}$ for the forward pass of backpropagation, so we'll store and return these as well.\n"
       ],
       "metadata": {
         "id": "5irtyxnLJSGX"
@@ -132,7 +131,7 @@
         "  K = len(all_weights) -1\n",
         "\n",
         "  # We'll store the pre-activations at each layer in a list \"all_f\"\n",
-        "  # and the activations in a second list[all_h].\n",
+        "  # and the activations in a second list \"all_h\".\n",
         "  all_f = [None] * (K+1)\n",
         "  all_h = [None] * (K+1)\n",
         "\n",
@@ -142,14 +141,14 @@
         "\n",
         "  # Run through the layers, calculating all_f[0...K-1] and all_h[1...K]\n",
         "  for layer in range(K):\n",
-        "      # Update preactivations and activations at this layer according to eqn 7.16\n",
-        "      # Remmember to use np.matmul for matrrix multiplications\n",
+        "      # Update preactivations and activations at this layer according to eqn 7.17\n",
+        "      # Remember to use np.matmul for matrix multiplications\n",
         "      # TODO -- Replace the lines below\n",
         "      all_f[layer] = all_h[layer]\n",
         "      all_h[layer+1] = all_f[layer]\n",
         "\n",
         "  # Compute the output from the last hidden layer\n",
-        "  # TO DO -- Replace the line below\n",
+        "  # TODO -- Replace the line below\n",
         "  all_f[K] = np.zeros_like(all_biases[-1])\n",
         "\n",
         "  # Retrieve the output\n",
@@ -166,7 +165,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Define in input\n",
+        "# Define input\n",
         "net_input = np.ones((D_i,1)) * 1.2\n",
         "# Compute network output\n",
         "net_output, all_f, all_h = compute_network_output(net_input,all_weights, all_biases)\n",
@@ -230,8 +229,8 @@
         "# We'll need the indicator function\n",
         "def indicator_function(x):\n",
         "  x_in = np.array(x)\n",
-        "  x_in[x_in>=0] = 1\n",
-        "  x_in[x_in<0] = 0\n",
+        "  x_in[x_in>0] = 1\n",
+        "  x_in[x_in<=0] = 0\n",
         "  return x_in\n",
         "\n",
         "# Main backward pass routine\n",
@@ -244,28 +243,28 @@
         "  all_dl_dh = [None] * (K+1)\n",
         "  # Again for convenience we'll stick with the convention that all_h[0] is the net input and all_f[k] in the net output\n",
         "\n",
-        "  # Compute derivatives of net output with respect to loss\n",
+        "  # Compute derivatives of the loss with respect to the network output\n",
         "  all_dl_df[K] = np.array(d_loss_d_output(all_f[K],y))\n",
         "\n",
         "  # Now work backwards through the network\n",
         "  for layer in range(K,-1,-1):\n",
-        "    # TODO Calculate the derivatives of biases at layer this from all_dl_df[layer]. (eq 7.21)\n",
+        "    # TODO Calculate the derivatives of the loss with respect to the biases at layer from all_dl_df[layer]. (eq 7.22)\n",
         "    # NOTE!  To take a copy of matrix X, use Z=np.array(X)\n",
         "    # REPLACE THIS LINE\n",
         "    all_dl_dbiases[layer] = np.zeros_like(all_biases[layer])\n",
         "\n",
-        "    # TODO Calculate the derivatives of weight at layer from all_dl_df[K] and all_h[K] (eq 7.22)\n",
+        "    # TODO Calculate the derivatives of the loss with respect to the weights at layer from all_dl_df[layer] and all_h[layer] (eq 7.23)\n",
         "    # Don't forget to use np.matmul\n",
         "    # REPLACE THIS LINE\n",
         "    all_dl_dweights[layer] = np.zeros_like(all_weights[layer])\n",
         "\n",
-        "    # TODO: calculate the derivatives of activations from weight and derivatives of next preactivations (eq 7.20)\n",
+        "    # TODO: calculate the derivatives of the loss with respect to the activations from weight and derivatives of next preactivations (second part of last line of eq 7.25)\n",
         "    # REPLACE THIS LINE\n",
         "    all_dl_dh[layer] = np.zeros_like(all_h[layer])\n",
         "\n",
         "\n",
         "    if layer > 0:\n",
-        "      # TODO Calculate the derivatives of the pre-activation f with respect to activation h (deriv of ReLu function)\n",
+        "      # TODO Calculate the derivatives of the loss with respect to the pre-activation f (use derivative of ReLu function, first part of last line of eq. 7.25)\n",
         "      # REPLACE THIS LINE\n",
         "      all_dl_df[layer-1] = np.zeros_like(all_f[layer-1])\n",
         "\n",
@@ -300,7 +299,7 @@
         "delta_fd = 0.000001\n",
         "\n",
         "# Test the dervatives of the bias vectors\n",
-        "for layer in range(K):\n",
+        "for layer in range(K+1):\n",
         "  dl_dbias  = np.zeros_like(all_dl_dbiases[layer])\n",
         "  # For every element in the bias\n",
         "  for row in range(all_biases[layer].shape[0]):\n",
@@ -311,14 +310,20 @@
         "    network_output_2, *_ = compute_network_output(net_input, all_weights, all_biases)\n",
         "    dl_dbias[row] = (least_squares_loss(network_output_1, y) - least_squares_loss(network_output_2,y))/delta_fd\n",
         "  all_dl_dbiases_fd[layer] = np.array(dl_dbias)\n",
+        "  print(\"-----------------------------------------------\")\n",
         "  print(\"Bias %d, derivatives from backprop:\"%(layer))\n",
         "  print(all_dl_dbiases[layer])\n",
         "  print(\"Bias %d, derivatives from finite differences\"%(layer))\n",
         "  print(all_dl_dbiases_fd[layer])\n",
+        "  if np.allclose(all_dl_dbiases_fd[layer],all_dl_dbiases[layer],rtol=1e-05, atol=1e-08, equal_nan=False):\n",
+        "    print(\"Success!  Derivatives match.\")\n",
+        "  else:\n",
+        "    print(\"Failure!  Derivatives different.\")\n",
+        "\n",
         "\n",
         "\n",
         "# Test the derivatives of the weights matrices\n",
-        "for layer in range(K):\n",
+        "for layer in range(K+1):\n",
         "  dl_dweight  = np.zeros_like(all_dl_dweights[layer])\n",
         "  # For every element in the bias\n",
         "  for row in range(all_weights[layer].shape[0]):\n",
@@ -330,10 +335,15 @@
         "      network_output_2, *_ = compute_network_output(net_input, all_weights, all_biases)\n",
         "      dl_dweight[row][col] = (least_squares_loss(network_output_1, y) - least_squares_loss(network_output_2,y))/delta_fd\n",
         "  all_dl_dweights_fd[layer] = np.array(dl_dweight)\n",
+        "  print(\"-----------------------------------------------\")\n",
         "  print(\"Weight %d, derivatives from backprop:\"%(layer))\n",
         "  print(all_dl_dweights[layer])\n",
         "  print(\"Weight %d, derivatives from finite differences\"%(layer))\n",
-        "  print(all_dl_dweights_fd[layer])"
+        "  print(all_dl_dweights_fd[layer])\n",
+        "  if np.allclose(all_dl_dweights_fd[layer],all_dl_dweights[layer],rtol=1e-05, atol=1e-08, equal_nan=False):\n",
+        "    print(\"Success!  Derivatives match.\")\n",
+        "  else:\n",
+        "    print(\"Failure!  Derivatives different.\")"
       ],
       "metadata": {
         "id": "PK-UtE3hreAK"
@@ -342,4 +352,4 @@
       "outputs": []
     }
   ]
-}
+}

Notebooks/Chap07/7_3_Initialization.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyNHLXFpiSnUzAbzhtOk+bxu",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +32,7 @@
         "\n",
         "This notebook explores weight initialization in deep neural networks as described in section 7.5 of the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -117,10 +116,10 @@
         "def compute_network_output(net_input, all_weights, all_biases):\n",
         "\n",
         "  # Retrieve number of layers\n",
-        "  K = len(all_weights) -1\n",
+        "  K = len(all_weights)-1\n",
         "\n",
         "  # We'll store the pre-activations at each layer in a list \"all_f\"\n",
-        "  # and the activations in a second list[all_h].\n",
+        "  # and the activations in a second list \"all_h\".\n",
         "  all_f = [None] * (K+1)\n",
         "  all_h = [None] * (K+1)\n",
         "\n",
@@ -151,7 +150,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Now let's investigate how this the size of the outputs vary as we change the initialization variance:\n"
+        "Now let's investigate how the size of the outputs vary as we change the initialization variance:\n"
       ],
       "metadata": {
         "id": "bIUrcXnOqChl"
@@ -164,7 +163,7 @@
         "K = 5\n",
         "# Number of neurons per layer\n",
         "D = 8\n",
-        "  # Input layer\n",
+        "# Input layer\n",
         "D_i = 1\n",
         "# Output layer\n",
         "D_o = 1\n",
@@ -177,7 +176,7 @@
         "data_in = np.random.normal(size=(1,n_data))\n",
         "net_output, all_f, all_h = compute_network_output(data_in, all_weights, all_biases)\n",
         "\n",
-        "for layer in range(K):\n",
+        "for layer in range(1,K+1):\n",
         "  print(\"Layer %d, std of hidden units = %3.3f\"%(layer, np.std(all_h[layer])))"
       ],
       "metadata": {
@@ -192,11 +191,11 @@
         "# You can see that the values of the hidden units are increasing on average (the variance is across all hidden units at the layer\n",
         "# and the 1000 training examples\n",
         "\n",
-        "# TO DO\n",
+        "# TODO\n",
         "# Change this to 50 layers with 80 hidden units per layer\n",
         "\n",
-        "# TO DO\n",
-        "# Now experiment with sigma_sq_omega to try to stop the variance of the forward computation explode"
+        "# TODO\n",
+        "# Now experiment with sigma_sq_omega to try to stop the variance of the forward computation exploding"
       ],
       "metadata": {
         "id": "VL_SO4tar3DC"
@@ -249,6 +248,9 @@
         "\n",
         "# Main backward pass routine\n",
         "def backward_pass(all_weights, all_biases, all_f, all_h, y):\n",
+        "  # Retrieve number of layers\n",
+        "  K = len(all_weights) - 1\n",
+        "\n",
         "  # We'll store the derivatives dl_dweights and dl_dbiases in lists as well\n",
         "  all_dl_dweights = [None] * (K+1)\n",
         "  all_dl_dbiases = [None] * (K+1)\n",
@@ -297,7 +299,7 @@
         "K = 5\n",
         "# Number of neurons per layer\n",
         "D = 8\n",
-        "  # Input layer\n",
+        "# Input layer\n",
         "D_i = 1\n",
         "# Output layer\n",
         "D_o = 1\n",
@@ -323,7 +325,7 @@
         "  for layer in range(1,K):\n",
         "    aggregate_dl_df[layer][:,c_data] = np.squeeze(all_dl_df[layer])\n",
         "\n",
-        "for layer in range(1,K):\n",
+        "for layer in reversed(range(1,K)):\n",
         "  print(\"Layer %d, std of dl_dh = %3.3f\"%(layer, np.std(aggregate_dl_df[layer].ravel())))\n"
       ],
       "metadata": {
@@ -335,13 +337,13 @@
     {
       "cell_type": "code",
       "source": [
-        "# You can see that the values of the hidden units are increasing on average (the variance is across all hidden units at the layer\n",
-        "# and the 1000 training examples\n",
+        "# You can see that the gradients of the hidden units are increasing on average (the standard deviation is across all hidden units at the layer\n",
+        "# and the 100 training examples\n",
         "\n",
-        "# TO DO\n",
+        "# TODO\n",
         "# Change this to 50 layers with 80 hidden units per layer\n",
         "\n",
-        "# TO DO\n",
+        "# TODO\n",
         "# Now experiment with sigma_sq_omega to try to stop the variance of the gradients exploding\n"
       ],
       "metadata": {

Notebooks/Chap08/8_1_MNIST_1D_Performance.ipynb

@@ -1,28 +1,10 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "gpuType": "T4",
-      "authorship_tag": "ABX9TyNLj3HOpVB87nRu7oSLuBaU",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    },
-    "accelerator": "GPU"
-  },
   "cells": [
     {
       "cell_type": "markdown",
       "metadata": {
-        "id": "view-in-github",
-        "colab_type": "text"
+        "colab_type": "text",
+        "id": "view-in-github"
       },
       "source": [
         "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/Notebooks/Chap08/8_1_MNIST_1D_Performance.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
@@ -30,33 +12,38 @@
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "L6chybAVFJW2"
+      },
       "source": [
         "# **Notebook 8.1: MNIST_1D_Performance**\n",
         "\n",
         "This notebook runs a simple neural network on the MNIST1D dataset as in figure 8.2a. It uses code from https://github.com/greydanus/mnist1d to generate the data.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
-      ],
-      "metadata": {
-        "id": "L6chybAVFJW2"
-      }
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
-        "!git clone https://github.com/greydanus/mnist1d"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "ifVjS4cTOqKz"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "# Run this if you're in a Colab to install MNIST 1D repository\n",
+        "%pip install git+https://github.com/greydanus/mnist1d"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "qyE7G1StPIqO"
+      },
+      "outputs": [],
       "source": [
         "import torch, torch.nn as nn\n",
         "from torch.utils.data import TensorDataset, DataLoader\n",
@@ -64,24 +51,24 @@
         "import numpy as np\n",
         "import matplotlib.pyplot as plt\n",
         "import mnist1d"
-      ],
-      "metadata": {
-        "id": "qyE7G1StPIqO"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Let's generate a training and test dataset using the MNIST1D code.  The dataset gets saved as a .pkl file so it doesn't have to be regenerated each time."
-      ],
       "metadata": {
         "id": "F7LNq72SP6jO"
-      }
+      },
+      "source": [
+        "Let's generate a training and test dataset using the MNIST1D code.  The dataset gets saved as a .pkl file so it doesn't have to be regenerated each time."
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "YLxf7dJfPaqw"
+      },
+      "outputs": [],
       "source": [
         "args = mnist1d.data.get_dataset_args()\n",
         "data = mnist1d.data.get_dataset(args, path='./mnist1d_data.pkl', download=False, regenerate=False)\n",
@@ -91,20 +78,20 @@
         "print(\"Examples in training set: {}\".format(len(data['y'])))\n",
         "print(\"Examples in test set: {}\".format(len(data['y_test'])))\n",
         "print(\"Length of each example: {}\".format(data['x'].shape[-1]))"
-      ],
-      "metadata": {
-        "id": "YLxf7dJfPaqw"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "FxaB5vc0uevl"
+      },
+      "outputs": [],
       "source": [
         "D_i = 40    # Input dimensions\n",
         "D_k = 100   # Hidden dimensions\n",
         "D_o = 10    # Output dimensions\n",
-        "# TO DO:\n",
+        "# TODO:\n",
         "# Define a model with two hidden layers of size 100\n",
         "# And ReLU activations between them\n",
         "# Replace this line (see Figure 7.8 of book for help):\n",
@@ -112,7 +99,7 @@
         "\n",
         "\n",
         "def weights_init(layer_in):\n",
-        "  # TO DO:\n",
+        "  # TODO:\n",
         "  # Initialize the parameters with He initialization\n",
         "  # Replace this line (see figure 7.8 of book for help)\n",
         "  print(\"Initializing layer\")\n",
@@ -120,15 +107,15 @@
         "\n",
         "# Call the function you just defined\n",
         "model.apply(weights_init)\n"
-      ],
-      "metadata": {
-        "id": "FxaB5vc0uevl"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "_rX6N3VyyQTY"
+      },
+      "outputs": [],
       "source": [
         "# choose cross entropy loss function (equation 5.24)\n",
         "loss_function = torch.nn.CrossEntropyLoss()\n",
@@ -136,11 +123,10 @@
         "optimizer = torch.optim.SGD(model.parameters(), lr = 0.05, momentum=0.9)\n",
         "# object that decreases learning rate by half every 10 epochs\n",
         "scheduler = StepLR(optimizer, step_size=10, gamma=0.5)\n",
-        "# create 100 dummy data points and store in data loader class\n",
         "x_train = torch.tensor(data['x'].astype('float32'))\n",
-        "y_train = torch.tensor(data['y'].transpose().astype('long'))\n",
+        "y_train = torch.tensor(data['y'].transpose().astype('int64'))\n",
         "x_test= torch.tensor(data['x_test'].astype('float32'))\n",
-        "y_test = torch.tensor(data['y_test'].astype('long'))\n",
+        "y_test = torch.tensor(data['y_test'].astype('int64'))\n",
         "\n",
         "# load the data into a class that creates the batches\n",
         "data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
@@ -185,15 +171,15 @@
         "\n",
         "  # tell scheduler to consider updating learning rate\n",
         "  scheduler.step()"
-      ],
-      "metadata": {
-        "id": "_rX6N3VyyQTY"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "yI-l6kA_EH9G"
+      },
+      "outputs": [],
       "source": [
         "# Plot the results\n",
         "fig, ax = plt.subplots()\n",
@@ -214,25 +200,38 @@
         "ax.set_title('Train loss %3.2f, Test loss %3.2f'%(losses_train[-1],losses_test[-1]))\n",
         "ax.legend()\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "yI-l6kA_EH9G"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "q-yT6re6GZS4"
+      },
       "source": [
-        "**TO DO**\n",
+        "**TODO**\n",
         "\n",
         "Play with the model -- try changing the number of layers, hidden units, learning rate, batch size, momentum or anything else you like.  See if you can improve the test results.\n",
         "\n",
         "Is it a good idea to optimize the hyperparameters in this way?  Will the final result be a good estimate of the true test performance?"
-      ],
-      "metadata": {
-        "id": "q-yT6re6GZS4"
-      }
+      ]
     }
-  ]
-}
+  ],
+  "metadata": {
+    "accelerator": "GPU",
+    "colab": {
+      "authorship_tag": "ABX9TyOuKMUcKfOIhIL2qTX9jJCy",
+      "gpuType": "T4",
+      "include_colab_link": true,
+      "provenance": []
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}

Notebooks/Chap08/8_2_Bias_Variance_Trade_Off.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyPz1B8kFc21JvGTDwqniloA",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +32,7 @@
         "\n",
         "This notebook investigates the bias-variance trade-off for the toy model used throughout chapter 8 and reproduces the bias/variance trade off curves seen in figure 8.9.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -77,7 +76,7 @@
         "    for i in range(n_data):\n",
         "        x[i] = np.random.uniform(i/n_data, (i+1)/n_data, 1)\n",
         "\n",
-        "    # y value from running through functoin and adding noise\n",
+        "    # y value from running through function and adding noise\n",
         "    y = np.ones(n_data)\n",
         "    for i in range(n_data):\n",
         "        y[i] = true_function(x[i])\n",
@@ -93,7 +92,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Draw the fitted function, together win uncertainty used to generate points\n",
+        "# Draw the fitted function, together with uncertainty used to generate points\n",
         "def plot_function(x_func, y_func, x_data=None,y_data=None, x_model = None, y_model =None, sigma_func = None, sigma_model=None):\n",
         "\n",
         "    fig,ax = plt.subplots()\n",
@@ -185,10 +184,8 @@
         "          if A[i,j] < 0:\n",
         "              A[i,j] = 0;\n",
         "\n",
-        "  ATA = np.matmul(np.transpose(A), A)\n",
-        "  ATAInv = np.linalg.inv(ATA)\n",
-        "  ATAInvAT = np.matmul(ATAInv, np.transpose(A))\n",
-        "  beta_omega = np.matmul(ATAInvAT,y)\n",
+        "  beta_omega = np.linalg.lstsq(A, y, rcond=None)[0]\n",
+        "\n",
         "  beta = beta_omega[0]\n",
         "  omega = beta_omega[1:]\n",
         "\n",
@@ -206,7 +203,7 @@
         "# Closed form solution\n",
         "beta, omega = fit_model_closed_form(x_data,y_data,n_hidden=3)\n",
         "\n",
-        "# Get prediction for model across graph grange\n",
+        "# Get prediction for model across graph range\n",
         "x_model = np.linspace(0,1,100);\n",
         "y_model = network(x_model, beta, omega)\n",
         "\n",
@@ -229,7 +226,7 @@
         "  y_model_all = np.zeros((n_datasets, x_model.shape[0]))\n",
         "\n",
         "  for c_dataset in range(n_datasets):\n",
-        "    # TODO -- Generate n_data x,y, pairs with standard divation sigma_func\n",
+        "    # TODO -- Generate n_data x,y, pairs with standard deviation sigma_func\n",
         "    # Replace this line\n",
         "    x_data,y_data = np.zeros([1,n_data]),np.zeros([1,n_data])\n",
         "\n",
@@ -271,7 +268,7 @@
         "mean_model, std_model = get_model_mean_variance(n_data, n_datasets, n_hidden, sigma_func) ;\n",
         "\n",
         "# Plot the results\n",
-        "plot_function(x_func, y_func, x_data,y_data, x_model, mean_model, sigma_model=std_model)"
+        "plot_function(x_func, y_func, x_model=x_model, y_model=mean_model, sigma_model=std_model)"
       ],
       "metadata": {
         "id": "Wxk64t2SoX9c"
@@ -296,7 +293,8 @@
       "cell_type": "code",
       "source": [
         "# Plot the noise, bias and variance as a function of capacity\n",
-        "hidden_variables = [1,2,3,4,5,6,7,8,9,10,11,12]\n",
+        "n_hidden = 12\n",
+        "hidden_variables = list(range(1, n_hidden + 1))\n",
         "bias = np.zeros((len(hidden_variables),1)) ;\n",
         "variance = np.zeros((len(hidden_variables),1)) ;\n",
         "\n",
@@ -305,7 +303,7 @@
         "sigma_func = 0.3\n",
         "n_hidden = 5\n",
         "\n",
-        "# Set random seed so that get same result every time\n",
+        "# Set random seed so that we get the same result every time\n",
         "np.random.seed(1)\n",
         "\n",
         "for c_hidden in range(len(hidden_variables)):\n",
@@ -316,7 +314,7 @@
         "\n",
         "  # Compute variance -- average of the model variance (average squared deviation of fitted models around mean fitted model)\n",
         "  variance[c_hidden] = 0\n",
-        "  # Compute bias (average squared deviaton of mean fitted model around true function)\n",
+        "  # Compute bias (average squared deviation of mean fitted model around true function)\n",
         "  bias[c_hidden] = 0\n",
         "\n",
         "# Plot the results\n",
@@ -324,7 +322,7 @@
         "ax.plot(hidden_variables, variance, 'k-')\n",
         "ax.plot(hidden_variables, bias, 'r-')\n",
         "ax.plot(hidden_variables, variance+bias, 'g-')\n",
-        "ax.set_xlim(0,12)\n",
+        "ax.set_xlim(0,n_hidden)\n",
         "ax.set_ylim(0,0.5)\n",
         "ax.set_xlabel(\"Model capacity\")\n",
         "ax.set_ylabel(\"Variance\")\n",
@@ -336,15 +334,6 @@
       },
       "execution_count": null,
       "outputs": []
-    },
-    {
-      "cell_type": "code",
-      "source": [],
-      "metadata": {
-        "id": "WKUyOAywL_b2"
-      },
-      "execution_count": null,
-      "outputs": []
     }
   ]
 }

Notebooks/Chap08/8_3_Double_Descent.ipynb

@@ -5,7 +5,6 @@
     "colab": {
       "provenance": [],
       "gpuType": "T4",
-      "authorship_tag": "ABX9TyN/KUpEObCKnHZ/4Onp5sHG",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -37,7 +36,7 @@
         "\n",
         "It uses the MNIST-1D database which can be found at https://github.com/greydanus/mnist1d\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -48,8 +47,8 @@
     {
       "cell_type": "code",
       "source": [
-        "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
-        "!git clone https://github.com/greydanus/mnist1d"
+        "# Run this if you're in a Colab to install MNIST 1D repository\n",
+        "!pip install git+https://github.com/greydanus/mnist1d"
       ],
       "metadata": {
         "id": "fn9BP5N5TguP"
@@ -100,7 +99,7 @@
         "# data['x'], data['y'], data['x_test'], and data['y_test']\n",
         "print(\"Examples in training set: {}\".format(len(data['y'])))\n",
         "print(\"Examples in test set: {}\".format(len(data['y_test'])))\n",
-        "print(\"Length of each example: {}\".format(data['x'].shape[-1]))"
+        "print(\"Dimensionality of each example: {}\".format(data['x'].shape[-1]))"
       ],
       "metadata": {
         "id": "PW2gyXL5UkLU"
@@ -124,7 +123,7 @@
         "  D_k = n_hidden   # Hidden dimensions\n",
         "  D_o = 10    # Output dimensions\n",
         "\n",
-        "  # Define a model with two hidden layers of size 100\n",
+        "  # Define a model with two hidden layers\n",
         "  # And ReLU activations between them\n",
         "  model = nn.Sequential(\n",
         "  nn.Linear(D_i, D_k),\n",
@@ -148,7 +147,7 @@
     {
       "cell_type": "code",
       "source": [
-        "def fit_model(model, data):\n",
+        "def fit_model(model, data, n_epoch):\n",
         "\n",
         "  # choose cross entropy loss function (equation 5.24)\n",
         "  loss_function = torch.nn.CrossEntropyLoss()\n",
@@ -157,7 +156,6 @@
         "  optimizer = torch.optim.SGD(model.parameters(), lr = 0.01, momentum=0.9)\n",
         "\n",
         "\n",
-        "  # create 100 dummy data points and store in data loader class\n",
         "  x_train = torch.tensor(data['x'].astype('float32'))\n",
         "  y_train = torch.tensor(data['y'].transpose().astype('long'))\n",
         "  x_test= torch.tensor(data['x_test'].astype('float32'))\n",
@@ -166,9 +164,6 @@
         "  # load the data into a class that creates the batches\n",
         "  data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
         "\n",
-        "  # loop over the dataset n_epoch times\n",
-        "  n_epoch = 1000\n",
-        "\n",
         "  for epoch in range(n_epoch):\n",
         "    # loop over batches\n",
         "    for i, batch in enumerate(data_loader):\n",
@@ -205,12 +200,24 @@
       "execution_count": null,
       "outputs": []
     },
+    {
+      "cell_type": "code",
+      "source": [
+        "def count_parameters(model):\n",
+        "    return sum(p.numel() for p in model.parameters() if p.requires_grad)"
+      ],
+      "metadata": {
+        "id": "AQNCmFNV6JpV"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
     {
       "cell_type": "markdown",
       "source": [
         "The following code produces the double descent curve by training the model with different numbers of hidden units and plotting the test error.\n",
         "\n",
-        "TO DO:\n",
+        "TODO:\n",
         "\n",
         "*Before* you run the code, and considering that there are 4000 training examples predict:<br>\n",
         "\n",
@@ -228,19 +235,27 @@
         "# This code will take a while (~30 mins on GPU) to run!  Go and make a cup of coffee!\n",
         "\n",
         "hidden_variables = np.array([2,4,6,8,10,14,18,22,26,30,35,40,45,50,55,60,70,80,90,100,120,140,160,180,200,250,300,400]) ;\n",
+        "\n",
         "errors_train_all = np.zeros_like(hidden_variables)\n",
         "errors_test_all = np.zeros_like(hidden_variables)\n",
+        "total_weights_all = np.zeros_like(hidden_variables)\n",
+        "\n",
+        "# loop over the dataset n_epoch times\n",
+        "n_epoch = 1000\n",
         "\n",
         "# For each hidden variable size\n",
         "for c_hidden in range(len(hidden_variables)):\n",
         "    print(f'Training model with {hidden_variables[c_hidden]:3d} hidden variables')\n",
         "    # Get a model\n",
         "    model = get_model(hidden_variables[c_hidden]) ;\n",
+        "    # Count and store number of weights\n",
+        "    total_weights_all[c_hidden] = count_parameters(model)\n",
         "    # Train the model\n",
-        "    errors_train, errors_test = fit_model(model, data)\n",
+        "    errors_train, errors_test = fit_model(model, data, n_epoch)\n",
         "    # Store the results\n",
         "    errors_train_all[c_hidden] = errors_train\n",
-        "    errors_test_all[c_hidden]= errors_test"
+        "    errors_test_all[c_hidden]= errors_test\n",
+        "\n"
       ],
       "metadata": {
         "id": "K4OmBZGHWXpk"
@@ -251,12 +266,29 @@
     {
       "cell_type": "code",
       "source": [
+        "import matplotlib.pyplot as plt\n",
+        "import numpy as np\n",
+        "\n",
+        "# Assuming data['y'] is available and contains the training examples\n",
+        "num_training_examples = len(data['y'])\n",
+        "\n",
+        "# Find the index where total_weights_all is closest to num_training_examples\n",
+        "closest_index = np.argmin(np.abs(np.array(total_weights_all) - num_training_examples))\n",
+        "\n",
+        "# Get the corresponding value of hidden variables\n",
+        "hidden_variable_at_num_training_examples = hidden_variables[closest_index]\n",
+        "\n",
         "# Plot the results\n",
         "fig, ax = plt.subplots()\n",
-        "ax.plot(hidden_variables, errors_train_all,'r-',label='train')\n",
-        "ax.plot(hidden_variables, errors_test_all,'b-',label='test')\n",
-        "ax.set_ylim(0,100);\n",
-        "ax.set_xlabel('No hidden variables'); ax.set_ylabel('Error')\n",
+        "ax.plot(hidden_variables, errors_train_all, 'r-', label='train')\n",
+        "ax.plot(hidden_variables, errors_test_all, 'b-', label='test')\n",
+        "\n",
+        "# Add a vertical line at the point where total weights equal the number of training examples\n",
+        "ax.axvline(x=hidden_variable_at_num_training_examples, color='g', linestyle='--', label='N(weights) = N(train)')\n",
+        "\n",
+        "ax.set_ylim(0, 100)\n",
+        "ax.set_xlabel('No. hidden variables')\n",
+        "ax.set_ylabel('Error')\n",
         "ax.legend()\n",
         "plt.show()\n"
       ],
@@ -265,6 +297,24 @@
       },
       "execution_count": null,
       "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [],
+      "metadata": {
+        "id": "KT4X8_hE5NFb"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [],
+      "metadata": {
+        "id": "iGKZSfVF2r4z"
+      },
+      "execution_count": null,
+      "outputs": []
     }
   ]
-}
+}

Notebooks/Chap08/8_4_High_Dimensional_Spaces.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyPXPDEQiwNw+kYhWfg4kjz6",
+      "authorship_tag": "ABX9TyPAKqlf9VxztHXKylyJwqe8",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +33,7 @@
         "\n",
         "This notebook investigates the strange properties of high-dimensional spaces as discussed in the notes at the end of chapter 8.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -134,7 +134,7 @@
       "source": [
         "# Volume of a hypersphere\n",
         "\n",
-        "In the second part of this notebook we calculate the volume of a hypersphere of radius 0.5 (i.e., of diameter 1) as a function of the radius.  Note that you you can check your answer by doing the calculation for 2D using the standard formula for the area of a circle and making sure it matches."
+        "In the second part of this notebook we calculate the volume of a hypersphere of radius 0.5 (i.e., of diameter 1) as a function of the radius.  Note that you can check your answer by doing the calculation for 2D using the standard formula for the area of a circle and making sure it matches."
       ],
       "metadata": {
         "id": "b2FYKV1SL4Z7"
@@ -145,7 +145,7 @@
       "source": [
         "def volume_of_hypersphere(diameter, dimensions):\n",
         "  # Formula given in Problem 8.7 of the book\n",
-        "  # You will need sci.special.gamma()\n",
+        "  # You will need sci.gamma()\n",
         "  # Check out:    https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.gamma.html\n",
         "  # Also use this value for pi\n",
         "  pi = np.pi\n",
@@ -224,7 +224,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "You should see see that by the time we get to 300 dimensions most of the volume is in the outer 1 percent. <br><br>\n",
+        "You should see that by the time we get to 300 dimensions most of the volume is in the outer 1 percent. <br><br>\n",
         "\n",
         "The conclusion of all of this is that in high dimensions you should be sceptical of your intuitions about how things work.  I have tried to visualize many things in one or two dimensions in the book, but you should also be sceptical about these visualizations!"
       ],
@@ -233,4 +233,4 @@
       }
     }
   ]
-}
+}

Notebooks/Chap09/9_1_L2_Regularization.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyPJzymRTuvoWggIskM2Kamc",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +32,7 @@
         "\n",
         "This notebook investigates adding L2 regularization to the loss function for the Gabor model as in figure 9.1.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
       ],
@@ -120,7 +119,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Initialize the parmaeters and draw the model\n",
+        "# Initialize the parameters and draw the model\n",
         "phi = np.zeros((2,1))\n",
         "phi[0] =  -5     # Horizontal offset\n",
         "phi[1] =  25     # Frequency\n",
@@ -178,7 +177,7 @@
         "\n",
         "def draw_loss_function(compute_loss, data,  model, my_colormap, phi_iters = None):\n",
         "\n",
-        "  # Make grid of intercept/slope values to plot\n",
+        "  # Make grid of offset/frequency values to plot\n",
         "  offsets_mesh, freqs_mesh = np.meshgrid(np.arange(-10,10.0,0.1), np.arange(2.5,22.5,0.1))\n",
         "  loss_mesh = np.zeros_like(freqs_mesh)\n",
         "  # Compute loss for every set of parameters\n",
@@ -304,7 +303,7 @@
         "for c_step in range (n_steps):\n",
         "  # Do gradient descent step\n",
         "  phi_all[:,c_step+1:c_step+2] = gradient_descent_step(phi_all[:,c_step:c_step+1],data, model)\n",
-        "  # Measure loss and draw model every 4th step\n",
+        "  # Measure loss and draw model every 8th step\n",
         "  if c_step % 8 == 0:\n",
         "    loss =  compute_loss(data[0,:], data[1,:], model, phi_all[:,c_step+1:c_step+2])\n",
         "    draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
@@ -369,7 +368,7 @@
         "# Code to draw the regularization function\n",
         "def draw_reg_function():\n",
         "\n",
-        "  # Make grid of intercept/slope values to plot\n",
+        "  # Make grid of offset/frequency values to plot\n",
         "  offsets_mesh, freqs_mesh = np.meshgrid(np.arange(-10,10.0,0.1), np.arange(2.5,22.5,0.1))\n",
         "  loss_mesh = np.zeros_like(freqs_mesh)\n",
         "  # Compute loss for every set of parameters\n",
@@ -399,7 +398,7 @@
         "# Code to draw loss function with regularization\n",
         "def draw_loss_function_reg(data,  model, lambda_, my_colormap, phi_iters = None):\n",
         "\n",
-        "  # Make grid of intercept/slope values to plot\n",
+        "  # Make grid of offset/frequency values to plot\n",
         "  offsets_mesh, freqs_mesh = np.meshgrid(np.arange(-10,10.0,0.1), np.arange(2.5,22.5,0.1))\n",
         "  loss_mesh = np.zeros_like(freqs_mesh)\n",
         "  # Compute loss for every set of parameters\n",
@@ -458,14 +457,14 @@
     {
       "cell_type": "code",
       "source": [
-        "def dldphi0(phi, lambda_):\n",
+        "def dregdphi0(phi, lambda_):\n",
         "  # TODO compute the derivative with respect to phi0\n",
         "  # Replace this line:]\n",
         "  deriv = 0\n",
         "\n",
         "  return deriv\n",
         "\n",
-        "def dldphi1(phi, lambda_):\n",
+        "def dregdphi1(phi, lambda_):\n",
         "  # TODO compute the derivative with respect to phi1\n",
         "  # Replace this line:]\n",
         "  deriv = 0\n",
@@ -475,8 +474,8 @@
         "\n",
         "\n",
         "def compute_gradient2(data_x, data_y, phi, lambda_):\n",
-        "    dl_dphi0 = gabor_deriv_phi0(data_x, data_y, phi[0],phi[1])+dldphi0(np.squeeze(phi), lambda_)\n",
-        "    dl_dphi1 = gabor_deriv_phi1(data_x, data_y, phi[0],phi[1])+dldphi1(np.squeeze(phi), lambda_)\n",
+        "    dl_dphi0 = gabor_deriv_phi0(data_x, data_y, phi[0],phi[1])+dregdphi0(np.squeeze(phi), lambda_)\n",
+        "    dl_dphi1 = gabor_deriv_phi1(data_x, data_y, phi[0],phi[1])+dregdphi1(np.squeeze(phi), lambda_)\n",
         "    # Return the gradient\n",
         "    return np.array([[dl_dphi0],[dl_dphi1]])\n",
         "\n",
@@ -512,7 +511,7 @@
         "for c_step in range (n_steps):\n",
         "  # Do gradient descent step\n",
         "  phi_all[:,c_step+1:c_step+2] = gradient_descent_step2(phi_all[:,c_step:c_step+1],lambda_, data, model)\n",
-        "  # Measure loss and draw model every 4th step\n",
+        "  # Measure loss and draw model every 8th step\n",
         "  if c_step % 8 == 0:\n",
         "    loss =  compute_loss2(data[0,:], data[1,:], model, phi_all[:,c_step+1:c_step+2], lambda_)\n",
         "    draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
@@ -528,11 +527,11 @@
     {
       "cell_type": "markdown",
       "source": [
-        "You should see that the gradient descent algorithm now finds the correct minimum.  By applying a tiny bit of domain knowledge (the parameter phi0 tends to be near zero and the parameters phi1 tends to be near 12.5), we get a better solution.  However, the cost is that this solution is slightly biased towards this prior knowledge."
+        "You should see that the gradient descent algorithm now finds the correct minimum.  By applying a tiny bit of domain knowledge (the parameter phi0 tends to be near zero and the parameter phi1 tends to be near 12.5), we get a better solution.  However, the cost is that this solution is slightly biased towards this prior knowledge."
       ],
       "metadata": {
         "id": "wrszSLrqZG4k"
       }
     }
   ]
-}
+}

Notebooks/Chap09/9_3_Ensembling.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyNuR7X+PMWRddy+WQr4gr5f",
+      "authorship_tag": "ABX9TyOAC7YLEqN5qZhJXqRj+aHB",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +33,7 @@
         "\n",
         "This notebook investigates how ensembling can improve the performance of models. We'll work with the simplified neural network model (figure 8.4 of book) which we can fit in closed form, and so we can eliminate any errors due to not finding the global maximum.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
       ],
@@ -52,7 +52,7 @@
         "# import libraries\n",
         "import numpy as np\n",
         "import matplotlib.pyplot as plt\n",
-        "# Define seed so get same results each time\n",
+        "# Define seed to get same results each time\n",
         "np.random.seed(1)"
       ]
     },
@@ -80,7 +80,7 @@
         "    for i in range(n_data):\n",
         "        x[i] = np.random.uniform(i/n_data, (i+1)/n_data, 1)\n",
         "\n",
-        "    # y value from running through functoin and adding noise\n",
+        "    # y value from running through function and adding noise\n",
         "    y = np.ones(n_data)\n",
         "    for i in range(n_data):\n",
         "        y[i] = true_function(x[i])\n",
@@ -96,7 +96,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Draw the fitted function, together win uncertainty used to generate points\n",
+        "# Draw the fitted function, together with uncertainty used to generate points\n",
         "def plot_function(x_func, y_func, x_data=None,y_data=None, x_model = None, y_model =None, sigma_func = None, sigma_model=None):\n",
         "\n",
         "    fig,ax = plt.subplots()\n",
@@ -137,7 +137,7 @@
         "n_data = 15\n",
         "x_data,y_data = generate_data(n_data, sigma_func)\n",
         "\n",
-        "# Plot the functinon, data and uncertainty\n",
+        "# Plot the function, data and uncertainty\n",
         "plot_function(x_func, y_func, x_data, y_data, sigma_func=sigma_func)"
       ],
       "metadata": {
@@ -184,7 +184,9 @@
         "  A = np.ones((n_data, n_hidden+1))\n",
         "  for i in range(n_data):\n",
         "      for j in range(1,n_hidden+1):\n",
+        "          # Compute preactivation\n",
         "          A[i,j] = x[i]-(j-1)/n_hidden\n",
+        "          # Apply the ReLU function\n",
         "          if A[i,j] < 0:\n",
         "              A[i,j] = 0;\n",
         "\n",
@@ -214,7 +216,7 @@
         "# Closed form solution\n",
         "beta, omega = fit_model_closed_form(x_data,y_data,n_hidden=14)\n",
         "\n",
-        "# Get prediction for model across graph grange\n",
+        "# Get prediction for model across graph range\n",
         "x_model = np.linspace(0,1,100);\n",
         "y_model = network(x_model, beta, omega)\n",
         "\n",
@@ -295,7 +297,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Plot the median of the results\n",
+        "# Plot the mean of the results\n",
         "# TODO -- find the mean prediction\n",
         "# Replace this line\n",
         "y_model_mean = all_y_model[0,:]\n",
@@ -323,4 +325,4 @@
       }
     }
   ]
-}
+}

Notebooks/Chap09/9_4_Bayesian_Approach.ipynb

@@ -1,20 +1,4 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "authorship_tag": "ABX9TyMB8B4269DVmrcLoCWrhzKF",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
       "cell_type": "markdown",
@@ -28,18 +12,18 @@
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "el8l05WQEO46"
+      },
       "source": [
         "# **Notebook 9.4: Bayesian approach**\n",
         "\n",
         "This notebook investigates the Bayesian approach to model fitting and reproduces figure 9.11 from the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
-      ],
-      "metadata": {
-        "id": "el8l05WQEO46"
-      }
+      ]
     },
     {
       "cell_type": "code",
@@ -52,26 +36,31 @@
         "# import libraries\n",
         "import numpy as np\n",
         "import matplotlib.pyplot as plt\n",
-        "# Define seed so get same results each time\n",
+        "# Define seed to get same results each time\n",
         "np.random.seed(1)"
       ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "3hpqmFyQNrbt"
+      },
+      "outputs": [],
       "source": [
         "# The true function that we are trying to estimate, defined on [0,1]\n",
         "def true_function(x):\n",
         "    y = np.exp(np.sin(x*(2*3.1413)))\n",
         "    return y"
-      ],
-      "metadata": {
-        "id": "3hpqmFyQNrbt"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "skZMM5TbNwq4"
+      },
+      "outputs": [],
       "source": [
         "# Generate some data points with or without noise\n",
         "def generate_data(n_data, sigma_y=0.3):\n",
@@ -80,23 +69,23 @@
         "    for i in range(n_data):\n",
         "        x[i] = np.random.uniform(i/n_data, (i+1)/n_data, 1)\n",
         "\n",
-        "    # y value from running through functoin and adding noise\n",
+        "    # y value from running through function and adding noise\n",
         "    y = np.ones(n_data)\n",
         "    for i in range(n_data):\n",
         "        y[i] = true_function(x[i])\n",
         "        y[i] += np.random.normal(0, sigma_y, 1)\n",
         "    return x,y"
-      ],
-      "metadata": {
-        "id": "skZMM5TbNwq4"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ziwD_R7lN0DY"
+      },
+      "outputs": [],
       "source": [
-        "# Draw the fitted function, together win uncertainty used to generate points\n",
+        "# Draw the fitted function, together with uncertainty used to generate points\n",
         "def plot_function(x_func, y_func, x_data=None,y_data=None, x_model = None, y_model =None, sigma_func = None, sigma_model=None):\n",
         "\n",
         "    fig,ax = plt.subplots()\n",
@@ -117,15 +106,15 @@
         "    ax.set_xlabel('Input, $x$')\n",
         "    ax.set_ylabel('Output, $y$')\n",
         "    plt.show()"
-      ],
-      "metadata": {
-        "id": "ziwD_R7lN0DY"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "2CgKanwaN3NM"
+      },
+      "outputs": [],
       "source": [
         "# Generate true function\n",
         "x_func = np.linspace(0, 1.0, 100)\n",
@@ -137,17 +126,17 @@
         "n_data = 15\n",
         "x_data,y_data = generate_data(n_data, sigma_func)\n",
         "\n",
-        "# Plot the functinon, data and uncertainty\n",
+        "# Plot the function, data and uncertainty\n",
         "plot_function(x_func, y_func, x_data, y_data, sigma_func=sigma_func)"
-      ],
-      "metadata": {
-        "id": "2CgKanwaN3NM"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "gorZ6i97N7AR"
+      },
+      "outputs": [],
       "source": [
         "# Define model -- beta is a scalar and omega has size n_hidden,1\n",
         "def network(x, beta, omega):\n",
@@ -165,15 +154,13 @@
         "    y = y + beta\n",
         "\n",
         "    return y"
-      ],
-      "metadata": {
-        "id": "gorZ6i97N7AR"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "i8T_QduzeBmM"
+      },
       "source": [
         "Now let's compute a probability distribution over the model parameters using Bayes's rule:\n",
         "\n",
@@ -184,69 +171,71 @@
         "We'll define the prior $Pr(\\boldsymbol\\phi)$ as:\n",
         "\n",
         "\\begin{equation}\n",
-        "Pr(\\boldsymbol\\phi) = \\mbox{Norm}_{\\boldsymbol\\phi}\\bigl[\\mathbf{0},\\sigma^2_p\\mathbf{I}\\bigr]\n",
+        "Pr(\\boldsymbol\\phi) = \\text{Norm}_{\\boldsymbol\\phi}\\bigl[\\mathbf{0},\\sigma^2_p\\mathbf{I}\\bigr]\n",
         "\\end{equation}\n",
         "\n",
         "where $\\phi=[\\omega_1,\\omega_2\\ldots \\omega_n, \\beta]^T$ and $\\sigma^2_{p}$  is the prior variance.\n",
         "\n",
         "The likelihood term $\\prod_{i=1}^{I} Pr(\\mathbf{y}_{i}|\\mathbf{x}_{i},\\boldsymbol\\phi)$ is given by:\n",
         "\n",
-        "\\begin{eqnarray}\n",
-        "\\prod_{i=1}^{I} Pr(\\mathbf{y}_{i}|\\mathbf{x}_{i},\\boldsymbol\\phi) &=& \\prod_{i=1}^{I} \\mbox{Norm}_{y_i}\\bigl[\\mbox{f}[\\mathbf{x}_{i},\\boldsymbol\\phi],\\sigma_d^2\\bigr]\\\\\n",
-        "&=& \\prod_{i=1}^{I} \\mbox{Norm}_{y_i}\\bigl[\\boldsymbol\\omega\\mathbf{h}_i+\\beta,\\sigma_d^2\\bigr]\\\\\n",
-        "&=& \\mbox{Norm}_{\\mathbf{y}}\\bigl[\\mathbf{H}^T\\boldsymbol\\phi,\\sigma^2\\mathbf{I}\\bigr].\n",
-        "\\end{eqnarray}\n",
+        "\\begin{align}\n",
+        "\\prod_{i=1}^{I} Pr(\\mathbf{y}_{i}|\\mathbf{x}_{i},\\boldsymbol\\phi) &=& \\prod_{i=1}^{I} \\text{Norm}_{y_i}\\bigl[\\text{f}[\\mathbf{x}_{i},\\boldsymbol\\phi],\\sigma_d^2\\bigr]\\\\\n",
+        "&=& \\prod_{i=1}^{I} \\text{Norm}_{y_i}\\bigl[\\boldsymbol\\omega\\mathbf{h}_i+\\beta,\\sigma_d^2\\bigr]\\\\\n",
+        "&=& \\text{Norm}_{\\mathbf{y}}\\bigl[\\mathbf{H}^T\\boldsymbol\\phi,\\sigma^2\\mathbf{I}\\bigr].\n",
+        "\\end{align}\n",
         "\n",
         "where $\\sigma^2$ is the measurement noise and $\\mathbf{h}_{i}$ is the column vector of hidden variables for the $i^{th}$ input.  Here the vector $\\mathbf{y}$ and matrix $\\mathbf{H}$ are defined as:\n",
         "\n",
         "\\begin{equation}\n",
-        "\\mathbf{y} = \\begin{bmatrix}y_1\\\\y_2\\\\\\vdots\\\\y_{I}\\end{bmatrix}\\quad\\mbox{and}\\quad \\mathbf{H} = \\begin{bmatrix}\\mathbf{h}_{1}&\\mathbf{h}_{2}&\\cdots&\\mathbf{h}_{I}\\\\1&1&\\cdots &1\\end{bmatrix}.\n",
+        "\\mathbf{y} = \\begin{bmatrix}y_1\\\\y_2\\\\\\vdots\\\\y_{I}\\end{bmatrix}\\quad\\text{and}\\quad \\mathbf{H} = \\begin{bmatrix}\\mathbf{h}_{1}&\\mathbf{h}_{2}&\\cdots&\\mathbf{h}_{I}\\\\1&1&\\cdots &1\\end{bmatrix}.\n",
         "\\end{equation}\n"
-      ],
-      "metadata": {
-        "id": "i8T_QduzeBmM"
-      }
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "JojV6ueRk49G"
+      },
       "source": [
         "To make progress we use the change of variable relation (Appendix C.3.4 of the book) to rewrite the likelihood term as a normal distribution in the parameters $\\boldsymbol\\phi$:\n",
         "\n",
-        "\\begin{eqnarray}\n",
+        "\\begin{align}\n",
         "\\prod_{i=1}^{I} Pr(\\mathbf{y}_{i}|\\mathbf{x}_{i},\\boldsymbol\\phi+\\beta)\n",
-        "&=& \\mbox{Norm}_{\\mathbf{y}}\\bigl[\\mathbf{H}^T\\boldsymbol\\phi,\\sigma^2\\bigr]\\\\\n",
-        "&\\propto& \\mbox{Norm}_{\\boldsymbol\\phi}\\bigl[(\\mathbf{H}\\mathbf{H}^T)^{-1}\\mathbf{H}\\mathbf{y},\\sigma^2(\\mathbf{H}\\mathbf{H}^t)^{-1}\\bigr]\n",
-        "\\end{eqnarray}\n"
-      ],
-      "metadata": {
-        "id": "JojV6ueRk49G"
-      }
+        "&=& \\text{Norm}_{\\mathbf{y}}\\bigl[\\mathbf{H}^T\\boldsymbol\\phi,\\sigma^2\\bigr]\\\\\n",
+        "&\\propto& \\text{Norm}_{\\boldsymbol\\phi}\\bigl[(\\mathbf{H}\\mathbf{H}^T)^{-1}\\mathbf{H}\\mathbf{y},\\sigma^2(\\mathbf{H}\\mathbf{H}^t)^{-1}\\bigr]\n",
+        "\\end{align}\n"
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "YX0O_Ciwp4W1"
+      },
       "source": [
         "Finally, we can combine the likelihood and prior terms using the product of two normal distributions relation (Appendix C.3.3).\n",
         "\n",
-        "\\begin{eqnarray}\n",
+        "\\begin{align}\n",
         " Pr(\\boldsymbol\\phi|\\{\\mathbf{x}_{i},\\mathbf{y}_{i}\\}) &\\propto& \\prod_{i=1}^{I} Pr(\\mathbf{y}_{i}|\\mathbf{x}_{i},\\boldsymbol\\phi) Pr(\\boldsymbol\\phi)\\\\\n",
-        " &\\propto&\\mbox{Norm}_{\\boldsymbol\\phi}\\bigl[(\\mathbf{H}\\mathbf{H}^T)^{-1}\\mathbf{H}\\mathbf{y},\\sigma^2(\\mathbf{H}\\mathbf{H}^T)^{-1}\\bigr] \\mbox{Norm}_{\\boldsymbol\\phi}\\bigl[\\mathbf{0},\\sigma^2_p\\mathbf{I}\\bigr]\\\\\n",
-        " &\\propto&\\mbox{Norm}_{\\boldsymbol\\phi}\\biggl[\\frac{1}{\\sigma^2}\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\mathbf{H}\\mathbf{y},\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\biggr].\n",
-        "\\end{eqnarray}\n",
+        " &\\propto&\\text{Norm}_{\\boldsymbol\\phi}\\bigl[(\\mathbf{H}\\mathbf{H}^T)^{-1}\\mathbf{H}\\mathbf{y},\\sigma^2(\\mathbf{H}\\mathbf{H}^T)^{-1}\\bigr] \\text{Norm}_{\\boldsymbol\\phi}\\bigl[\\mathbf{0},\\sigma^2_p\\mathbf{I}\\bigr]\\\\\n",
+        " &\\propto&\\text{Norm}_{\\boldsymbol\\phi}\\biggl[\\frac{1}{\\sigma^2}\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\mathbf{H}\\mathbf{y},\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\biggr].\n",
+        "\\end{align}\n",
         "\n",
-        "In fact, since this already a normal distribution, the constant of proportionality must be one and we can write\n",
+        "In fact, since this is already a normal distribution, the constant of proportionality must be one and we can write\n",
         "\n",
-        "\\begin{eqnarray}\n",
-        " Pr(\\boldsymbol\\phi|\\{\\mathbf{x}_{i},\\mathbf{y}_{i}\\}) &=& \\mbox{Norm}_{\\boldsymbol\\phi}\\biggl[\\frac{1}{\\sigma^2}\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\mathbf{H}\\mathbf{y},\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\biggr].\n",
-        "\\end{eqnarray}\n",
+        "\\begin{align}\n",
+        " Pr(\\boldsymbol\\phi|\\{\\mathbf{x}_{i},\\mathbf{y}_{i}\\}) &=& \\text{Norm}_{\\boldsymbol\\phi}\\biggl[\\frac{1}{\\sigma^2}\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\mathbf{H}\\mathbf{y},\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\biggr].\n",
+        "\\end{align}\n",
         "\n",
         "TODO -- On a piece of paper, use the relations in Appendix C.3.3 and C.3.4 to fill in the missing steps and establish that this is the correct formula for the posterior."
-      ],
-      "metadata": {
-        "id": "YX0O_Ciwp4W1"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "nF1AcgNDwm4t"
+      },
+      "outputs": [],
       "source": [
         "def compute_H(x_data, n_hidden):\n",
         "  psi1 = np.ones((n_hidden+1,1));\n",
@@ -280,24 +269,24 @@
         "\n",
         "\n",
         "  return phi_mean, phi_covar"
-      ],
-      "metadata": {
-        "id": "nF1AcgNDwm4t"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Now we can draw samples from this distribution"
-      ],
       "metadata": {
         "id": "GjPnlG4q0UFK"
-      }
+      },
+      "source": [
+        "Now we can draw samples from this distribution"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "K4vYc82D0BMq"
+      },
+      "outputs": [],
       "source": [
         "# Define parameters\n",
         "n_hidden = 5\n",
@@ -313,15 +302,15 @@
         "x_model = x_func\n",
         "y_model_mean = network(x_model, phi_mean[-1], phi_mean[0:n_hidden])\n",
         "plot_function(x_func, y_func, x_data, y_data, x_model, y_model_mean)"
-      ],
-      "metadata": {
-        "id": "K4vYc82D0BMq"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "TVIjhubkSw-R"
+      },
+      "outputs": [],
       "source": [
         "# TODO Draw two samples from the normal distribution over the parameters\n",
         "# Replace these lines\n",
@@ -336,37 +325,40 @@
         "# Draw the two models\n",
         "plot_function(x_func, y_func, x_data, y_data, x_model, y_model_sample1)\n",
         "plot_function(x_func, y_func, x_data, y_data, x_model, y_model_sample2)"
-      ],
-      "metadata": {
-        "id": "TVIjhubkSw-R"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "GiNg5EroUiUb"
+      },
       "source": [
         "Now we need to perform inference for a new data points $\\mathbf{x}^*$ with corresponding hidden values $\\mathbf{h}^*$.  Instead of having a single estimate of the parameters, we have a distribution over the possible parameters.  So we marginalize (integrate) over this distribution to account for all possible values:\n",
         "\n",
-        "\\begin{eqnarray}\n",
-        "Pr(y^*|\\mathbf{x}^*)  &=& \\int Pr(y^{*}|\\mathbf{x}^*,\\boldsymbol\\phi)Pr(\\boldsymbol\\phi|\\{\\mathbf{x}_{i},\\mathbf{y}_{i}\\}) d\\boldsymbol\\phi\\\\\n",
-        "&=& \\int \\mbox{Norm}_{y^*}\\bigl[\\begin{bmatrix}\\mathbf{h}^{*T}&1\\end{bmatrix}\\boldsymbol\\phi,\\sigma^2]\\cdot\\mbox{Norm}_{\\boldsymbol\\phi}\\biggl[\\frac{1}{\\sigma^2}\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\mathbf{H}\\mathbf{y},\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\biggr]d\\boldsymbol\\phi\\\\\n",
-        "&=& \\mbox{Norm}_{y^*}\\biggl[\\frac{1}{\\sigma^2} \\begin{bmatrix}\\mathbf{h}^{*T}&1\\end{bmatrix}\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\mathbf{H}\\mathbf{y},  \\begin{bmatrix}\\mathbf{h}^{*T}&1\\end{bmatrix}\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\n",
-        "\\begin{bmatrix}\\mathbf{h}^*\\\\1\\end{bmatrix}\\biggr]\n",
-        "\\end{eqnarray}\n",
+        "\\begin{align}\n",
+        "Pr(y^*|\\mathbf{x}^*)  &= \\int Pr(y^{*}|\\mathbf{x}^*,\\boldsymbol\\phi)Pr(\\boldsymbol\\phi|\\{\\mathbf{x}_{i},\\mathbf{y}_{i}\\}) d\\boldsymbol\\phi\\\\\n",
+        "&= \\int \\text{Norm}_{y^*}\\bigl[[\\mathbf{h}^{*T},1]\\boldsymbol\\phi,\\sigma^2\\bigr]\\cdot\\text{Norm}_{\\boldsymbol\\phi}\\biggl[\\frac{1}{\\sigma^2}\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\mathbf{H}\\mathbf{y},\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\biggr]d\\boldsymbol\\phi\\\\\n",
+        "&= \\text{Norm}_{y^*}\\biggl[\\frac{1}{\\sigma^2} [\\mathbf{h}^{*T},1]\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\mathbf{H}\\mathbf{y},  [\\mathbf{h}^{*T},1]\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\n",
+        "[\\mathbf{h}^*;1]\\biggr],\n",
+        "\\end{align}\n",
         "\n",
-        "To compute this, we reformulated the integrand using the relations from appendices\n",
-        "C.3.3 and C.3.4 as the product of a normal distribution in $\\boldsymbol\\phi$ and a constant with respect\n",
-        "to $\\boldsymbol\\phi$. The integral of the normal distribution must be one, and so the finnal result is just the constant. This constant is itself a normal distribution in $y^*$. <br>\n",
+        "where the notation $[\\mathbf{h}^{*T},1]$ is a row vector containing $\\mathbf{h}^{*T}$ with a one appended to the end and $[\\mathbf{h}^{*};1 ]$ is a column vector containing $\\mathbf{h}^{*}$ with a one appended to the end.\n",
         "\n",
-        "If you feel so inclined you can work through the math of this yourself."
-      ],
-      "metadata": {
-        "id": "GiNg5EroUiUb"
-      }
+        "\n",
+        "To compute this, we reformulated the integrand using the relations from appendices C.3.3 and C.3.4 as the product of a normal distribution in $\\boldsymbol\\phi$ and a constant with respect\n",
+        "to $\\boldsymbol\\phi$. The integral of the normal distribution must be one, and so the final result is just the constant. This constant is itself a normal distribution in $y^*$. <br>\n",
+        "\n",
+        "If you feel so inclined you can work through the math of this yourself.\n",
+        "\n"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ILxT4EfW2lUm"
+      },
+      "outputs": [],
       "source": [
         "# Predict mean and variance of y_star from x_star\n",
         "def inference(x_star, x_data, y_data, sigma_sq, sigma_p_sq, n_hidden):\n",
@@ -381,15 +373,15 @@
         "  y_star_var =  1\n",
         "\n",
         "  return y_star_mean, y_star_var"
-      ],
-      "metadata": {
-        "id": "ILxT4EfW2lUm"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "87cjUjMaixHZ"
+      },
+      "outputs": [],
       "source": [
         "x_model = x_func\n",
         "y_model = np.zeros_like(x_model)\n",
@@ -401,24 +393,34 @@
         "\n",
         "# Draw the model\n",
         "plot_function(x_func, y_func, x_data, y_data, x_model, y_model, sigma_model=y_model_std)\n"
-      ],
-      "metadata": {
-        "id": "87cjUjMaixHZ"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "8Hcbe_16sK0F"
+      },
       "source": [
         "TODO:\n",
         "\n",
         "1.  Experiment running this again with different numbers of hidden units.  Make a prediction for what will happen when you increase / decrease them.\n",
         "2.  Experiment with what happens if you make the prior variance $\\sigma^2_p$ to a smaller value like 1.  How do you explain the results?"
-      ],
-      "metadata": {
-        "id": "8Hcbe_16sK0F"
-      }
+      ]
     }
-  ]
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
 }

Notebooks/Chap09/9_5_Augmentation.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyM3wq9CHLjekkIXIgXRxueE",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +32,7 @@
         "\n",
         "This notebook investigates data augmentation for the MNIST-1D model.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
       ],
@@ -44,8 +43,8 @@
     {
       "cell_type": "code",
       "source": [
-        "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
-        "!git clone https://github.com/greydanus/mnist1d"
+        "# Run this if you're in a Colab to install MNIST 1D repository\n",
+        "!pip install git+https://github.com/greydanus/mnist1d"
       ],
       "metadata": {
         "id": "syvgxgRr3myY"
@@ -95,7 +94,7 @@
         "D_k = 200   # Hidden dimensions\n",
         "D_o = 10    # Output dimensions\n",
         "\n",
-        "# Define a model with two hidden layers of size 100\n",
+        "# Define a model with two hidden layers of size 200\n",
         "# And ReLU activations between them\n",
         "model = nn.Sequential(\n",
         "nn.Linear(D_i, D_k),\n",
@@ -108,10 +107,7 @@
         "  # Initialize the parameters with He initialization\n",
         "  if isinstance(layer_in, nn.Linear):\n",
         "    nn.init.kaiming_uniform_(layer_in.weight)\n",
-        "    layer_in.bias.data.fill_(0.0)\n",
-        "\n",
-        "# Call the function you just defined\n",
-        "model.apply(weights_init)"
+        "    layer_in.bias.data.fill_(0.0)\n"
       ],
       "metadata": {
         "id": "JfIFWFIL33eF"
@@ -186,7 +182,7 @@
         "ax.plot(errors_test,'b-',label='test')\n",
         "ax.set_ylim(0,100); ax.set_xlim(0,n_epoch)\n",
         "ax.set_xlabel('Epoch'); ax.set_ylabel('Error')\n",
-        "ax.set_title('TrainError %3.2f, Test Error %3.2f'%(errors_train[-1],errors_test[-1]))\n",
+        "ax.set_title('Train Error %3.2f, Test Error %3.2f'%(errors_train[-1],errors_test[-1]))\n",
         "ax.legend()\n",
         "plt.show()"
       ],
@@ -208,14 +204,14 @@
     {
       "cell_type": "code",
       "source": [
-        "def augment(data_in):\n",
+        "def augment(input_vector):\n",
         "  # Create output vector\n",
-        "  data_out = np.zeros_like(data_in)\n",
+        "  data_out = np.zeros_like(input_vector)\n",
         "\n",
         "  # TODO:  Shift the input data by a random offset\n",
         "  # (rotating, so points that would go off the end, are added back to the beginning)\n",
         "  # Replace this line:\n",
-        "  data_out = np.zeros_like(data_in) ;\n",
+        "  data_out = np.zeros_like(input_vector) ;\n",
         "\n",
         "  # TODO:    # Randomly scale the data by a factor drawn from a uniform distribution over [0.8,1.2]\n",
         "  # Replace this line:\n",
@@ -233,7 +229,7 @@
       "cell_type": "code",
       "source": [
         "n_data_orig = data['x'].shape[0]\n",
-        "# We'll double the amount o fdata\n",
+        "# We'll double the amount of data\n",
         "n_data_augment = n_data_orig+4000\n",
         "augmented_x = np.zeros((n_data_augment, D_i))\n",
         "augmented_y = np.zeros(n_data_augment)\n",

Notebooks/Chap10/10_1_1D_Convolution.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyPHUNRkJMI5LujaxIXNV60m",
+      "authorship_tag": "ABX9TyML7rfAGE4gvmNUEiK5x3PS",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +33,7 @@
         "\n",
         "This notebook investigates 1D convolutional layers.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
       ],
@@ -41,6 +41,17 @@
         "id": "el8l05WQEO46"
       }
     },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "NOTE!!\n",
+        "\n",
+        "If you have the first edition of the printed book, it mistakenly refers to a convolutional filter with no spaces between the elements (i.e. a normal filter without dilation) as having dilation zero.  Actually, the convention is (weirdly) that this has dilation one.  And when there is one space between the elements, this is dilation two.   This notebook reflects the correct convention and so will be out of sync with the printed book.  If this is confusing, check the [errata](https://github.com/udlbook/udlbook/blob/main/UDL_Errata.pdf) document."
+      ],
+      "metadata": {
+        "id": "ggQrHkFZcUiV"
+      }
+    },
     {
       "cell_type": "code",
       "source": [
@@ -50,7 +61,7 @@
       "metadata": {
         "id": "nw7k5yCtOzoK"
       },
-      "execution_count": null,
+      "execution_count": 1,
       "outputs": []
     },
     {
@@ -85,10 +96,10 @@
       "cell_type": "code",
       "source": [
         "# Now let's define a zero-padded convolution operation\n",
-        "# with a convolution kernel size of 3, a stride of 1, and a dilation of 0\n",
+        "# with a convolution kernel size of 3, a stride of 1, and a dilation of 1\n",
         "# as in figure 10.2a-c.  Write it yourself, don't call a library routine!\n",
         "# Don't forget that Python arrays are indexed from zero, not from 1 as in the book figures\n",
-        "def conv_3_1_0_zp(x_in, omega):\n",
+        "def conv_3_1_1_zp(x_in, omega):\n",
         "    x_out = np.zeros_like(x_in)\n",
         "    # TODO -- write this function\n",
         "    # replace this line\n",
@@ -119,7 +130,7 @@
       "source": [
         "\n",
         "omega = [0.33,0.33,0.33]\n",
-        "h = conv_3_1_0_zp(x, omega)\n",
+        "h = conv_3_1_1_zp(x, omega)\n",
         "\n",
         "# Check that you have computed this correctly\n",
         "print(f\"Sum of output is {np.sum(h):3.3}, should be 71.1\")\n",
@@ -155,7 +166,7 @@
       "source": [
         "\n",
         "omega = [-0.5,0,0.5]\n",
-        "h2 = conv_3_1_0_zp(x, omega)\n",
+        "h2 = conv_3_1_1_zp(x, omega)\n",
         "\n",
         "# Draw the signal\n",
         "fig,ax = plt.subplots()\n",
@@ -187,9 +198,9 @@
       "cell_type": "code",
       "source": [
         "# Now let's define a zero-padded convolution operation\n",
-        "# with a convolution kernel size of 3, a stride of 2, and a dilation of 0\n",
-        "# as in figure 10.2a-c.  Write it yourself, don't call a library routine!\n",
-        "def conv_3_2_0_zp(x_in, omega):\n",
+        "# with a convolution kernel size of 3, a stride of 2, and a dilation of 1\n",
+        "# as in figure 10.3a-b.  Write it yourself, don't call a library routine!\n",
+        "def conv_3_2_1_zp(x_in, omega):\n",
         "    x_out = np.zeros(int(np.ceil(len(x_in)/2)))\n",
         "    # TODO -- write this function\n",
         "    # replace this line\n",
@@ -209,7 +220,7 @@
       "cell_type": "code",
       "source": [
         "omega = [0.33,0.33,0.33]\n",
-        "h3 = conv_3_2_0_zp(x, omega)\n",
+        "h3 = conv_3_2_1_zp(x, omega)\n",
         "\n",
         "# If you have done this right, the output length should be six and it should\n",
         "# contain every other value from the original convolution with stride 1\n",
@@ -226,9 +237,9 @@
       "cell_type": "code",
       "source": [
         "# Now let's define a zero-padded convolution operation\n",
-        "# with a convolution kernel size of 5, a stride of 1, and a dilation of 0\n",
-        "# as in figure 10.2a-c.  Write it yourself, don't call a library routine!\n",
-        "def conv_5_1_0_zp(x_in, omega):\n",
+        "# with a convolution kernel size of 5, a stride of 1, and a dilation of 1\n",
+        "# as in figure 10.3c.  Write it yourself, don't call a library routine!\n",
+        "def conv_5_1_1_zp(x_in, omega):\n",
         "    x_out = np.zeros_like(x_in)\n",
         "    # TODO -- write this function\n",
         "    # replace this line\n",
@@ -249,7 +260,7 @@
       "source": [
         "\n",
         "omega2 = [0.2, 0.2, 0.2, 0.2, 0.2]\n",
-        "h4 = conv_5_1_0_zp(x, omega2)\n",
+        "h4 = conv_5_1_1_zp(x, omega2)\n",
         "\n",
         "# Check that you have computed this correctly\n",
         "print(f\"Sum of output is {np.sum(h4):3.3}, should be 69.6\")\n",
@@ -273,10 +284,10 @@
       "cell_type": "code",
       "source": [
         "# Finally let's define a zero-padded convolution operation\n",
-        "# with a convolution kernel size of 3, a stride of 1, and a dilation of 1\n",
-        "# as in figure 10.2a-c.  Write it yourself, don't call a library routine!\n",
+        "# with a convolution kernel size of 3, a stride of 1, and a dilation of 2\n",
+        "# as in figure 10.3d.  Write it yourself, don't call a library routine!\n",
         "# Don't forget that Python arrays are indexed from zero, not from 1 as in the book figures\n",
-        "def conv_3_1_1_zp(x_in, omega):\n",
+        "def conv_3_1_2_zp(x_in, omega):\n",
         "    x_out = np.zeros_like(x_in)\n",
         "    # TODO -- write this function\n",
         "    # replace this line\n",
@@ -295,7 +306,7 @@
       "cell_type": "code",
       "source": [
         "omega = [0.33,0.33,0.33]\n",
-        "h5 = conv_3_1_1_zp(x, omega)\n",
+        "h5 = conv_3_1_2_zp(x, omega)\n",
         "\n",
         "# Check that you have computed this correctly\n",
         "print(f\"Sum of output is {np.sum(h5):3.3}, should be 68.3\")\n",
@@ -328,9 +339,9 @@
       "cell_type": "code",
       "source": [
         "# Compute matrix in figure 10.4 d\n",
-        "def get_conv_mat_3_1_0_zp(n_out, omega):\n",
+        "def get_conv_mat_3_1_1_zp(n_out, omega):\n",
         "  omega_mat = np.zeros((n_out,n_out))\n",
-        "  # TODO Fill in this matix\n",
+        "  # TODO Fill in this matrix\n",
         "  # Replace this line:\n",
         "  omega_mat = omega_mat\n",
         "\n",
@@ -349,11 +360,11 @@
       "source": [
         "# Run original convolution\n",
         "omega = np.array([-1.0,0.5,-0.2])\n",
-        "h6 = conv_3_1_0_zp(x, omega)\n",
+        "h6 = conv_3_1_1_zp(x, omega)\n",
         "print(h6)\n",
         "\n",
         "# If you have done this right, you should get the same answer\n",
-        "omega_mat = get_conv_mat_3_1_0_zp(len(x), omega)\n",
+        "omega_mat = get_conv_mat_3_1_1_zp(len(x), omega)\n",
         "h7 = np.matmul(omega_mat, x)\n",
         "print(h7)\n"
       ],
@@ -373,4 +384,4 @@
       }
     }
   ]
-}
+}

Notebooks/Chap10/10_2_Convolution_for_MNIST_1D.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyOgDisWDe/zHpfTGCH8AZ3i",
+      "authorship_tag": "ABX9TyNb46PJB/CC1pcHGfjpUUZg",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +33,7 @@
         "\n",
         "This notebook investigates a 1D convolutional network for MNIST-1D as in figure 10.7 and 10.8a.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
         "\n"
@@ -45,8 +45,8 @@
     {
       "cell_type": "code",
       "source": [
-        "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
-        "!git clone https://github.com/greydanus/mnist1d"
+        "# Run this if you're in a Colab to install MNIST 1D repository\n",
+        "!pip install git+https://github.com/greydanus/mnist1d"
       ],
       "metadata": {
         "id": "D5yLObtZCi9J"
@@ -128,11 +128,11 @@
         "\n",
         "\n",
         "# TODO Create a model with the following layers\n",
-        "# 1. Convolutional layer, (input=length 40 and 1 channel, kernel size 3x3, stride 2, padding=\"valid\", 15 output channels )\n",
+        "# 1. Convolutional layer, (input=length 40 and 1 channel, kernel size 3, stride 2, padding=\"valid\", 15 output channels )\n",
         "# 2. ReLU\n",
-        "# 3. Convolutional layer, (input=length 19 and 15 channels, kernel size 3x3, stride 2, padding=\"valid\", 15 output channels )\n",
+        "# 3. Convolutional layer, (input=length 19 and 15 channels, kernel size 3, stride 2, padding=\"valid\", 15 output channels )\n",
         "# 4. ReLU\n",
-        "# 5. Convolutional layer, (input=length 9 and 15 channels, kernel size 3x3, stride 2, padding=\"valid\", 15 output channels)\n",
+        "# 5. Convolutional layer, (input=length 9 and 15 channels, kernel size 3, stride 2, padding=\"valid\", 15 output channels)\n",
         "# 6. ReLU\n",
         "# 7. Flatten (converts 4x15) to length 60\n",
         "# 8. Linear layer (input size = 60, output size = 10)\n",
@@ -141,6 +141,9 @@
         "# https://pytorch.org/docs/stable/generated/torch.nn.Flatten.html\n",
         "# https://pytorch.org/docs/1.13/generated/torch.nn.Linear.html?highlight=linear#torch.nn.Linear\n",
         "\n",
+        "# NOTE THAT THE CONVOLUTIONAL LAYERS NEED TO TAKE THE NUMBER OF INPUT CHANNELS AS A PARAMETER\n",
+        "# AND NOT THE INPUT SIZE.\n",
+        "\n",
         "# Replace the following function:\n",
         "model = nn.Sequential(\n",
         "nn.Flatten(),\n",
@@ -185,9 +188,9 @@
         "scheduler = StepLR(optimizer, step_size=20, gamma=0.5)\n",
         "# create 100 dummy data points and store in data loader class\n",
         "x_train = torch.tensor(train_data_x.transpose().astype('float32'))\n",
-        "y_train = torch.tensor(train_data_y.astype('long'))\n",
+        "y_train = torch.tensor(train_data_y.astype('long')).long()\n",
         "x_val= torch.tensor(val_data_x.transpose().astype('float32'))\n",
-        "y_val = torch.tensor(val_data_y.astype('long'))\n",
+        "y_val = torch.tensor(val_data_y.astype('long')).long()\n",
         "\n",
         "# load the data into a class that creates the batches\n",
         "data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",

Notebooks/Chap10/10_3_2D_Convolution.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyMmbD0cKYvIHXbKX4AupA1x",
+      "authorship_tag": "ABX9TyNDaU2KKZDyY9Ea7vm/fNxo",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +33,7 @@
         "\n",
         "This notebook investigates the 2D convolution operation.  It asks you to hand code the convolution so we can be sure that we are computing the same thing as in PyTorch.  The next notebook uses the convolutional layers in PyTorch directly.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -114,6 +114,11 @@
         "    # Create output\n",
         "    out = np.zeros((batchSize, channelsOut, imageHeightOut, imageWidthOut), dtype=np.float32)\n",
         "\n",
+        "    # !!!!!! NOTE THERE IS A SUBTLETY HERE !!!!!!!!\n",
+        "    # I have padded the image with zeros above, so it is surrouned by a \"ring\" of zeros\n",
+        "    # That means that the image indexes are all off by one\n",
+        "    # This actually makes your code simpler\n",
+        "\n",
         "    for c_y in range(imageHeightOut):\n",
         "      for c_x in range(imageWidthOut):\n",
         "        for c_kernel_y in range(kernelHeight):\n",

Notebooks/Chap10/10_4_Downsampling_and_Upsampling.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyMbSR8fzpXvO6TIQdO7bI0H",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -31,9 +30,9 @@
       "source": [
         "# **Notebook 10.4: Downsampling and Upsampling**\n",
         "\n",
-        "This notebook investigates the down sampling and downsampling methods discussed in section 10.4 of the book.\n",
+        "This notebook investigates the upsampling and downsampling methods discussed in section 10.4 of the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
       ],
@@ -71,9 +70,9 @@
     {
       "cell_type": "code",
       "source": [
-        "def subsample(x_in):\n",
+        "def downsample(x_in):\n",
         "  x_out = np.zeros(( int(np.ceil(x_in.shape[0]/2)), int(np.ceil(x_in.shape[1]/2)) ))\n",
-        "  # TO DO -- write the subsampling routine\n",
+        "  # TODO -- write the downsampling routine\n",
         "  # Replace this line\n",
         "  x_out = x_out\n",
         "\n",
@@ -91,8 +90,8 @@
       "source": [
         "print(\"Original:\")\n",
         "print(orig_4_4)\n",
-        "print(\"Subsampled:\")\n",
-        "print(subsample(orig_4_4))"
+        "print(\"Downsampled:\")\n",
+        "print(downsample(orig_4_4))"
       ],
       "metadata": {
         "id": "O_i0y72_JwGZ"
@@ -127,24 +126,24 @@
         "image = Image.open('test_image.png')\n",
         "# convert image to numpy array\n",
         "data = asarray(image)\n",
-        "data_subsample = subsample(data);\n",
+        "data_downsample = downsample(data);\n",
         "\n",
         "plt.figure(figsize=(5,5))\n",
         "plt.imshow(data, cmap='gray')\n",
         "plt.show()\n",
         "\n",
         "plt.figure(figsize=(5,5))\n",
-        "plt.imshow(data_subsample, cmap='gray')\n",
+        "plt.imshow(data_downsample, cmap='gray')\n",
         "plt.show()\n",
         "\n",
-        "data_subsample2 = subsample(data_subsample)\n",
+        "data_downsample2 = downsample(data_downsample)\n",
         "plt.figure(figsize=(5,5))\n",
-        "plt.imshow(data_subsample2, cmap='gray')\n",
+        "plt.imshow(data_downsample2, cmap='gray')\n",
         "plt.show()\n",
         "\n",
-        "data_subsample3 = subsample(data_subsample2)\n",
+        "data_downsample3 = downsample(data_downsample2)\n",
         "plt.figure(figsize=(5,5))\n",
-        "plt.imshow(data_subsample3, cmap='gray')\n",
+        "plt.imshow(data_downsample3, cmap='gray')\n",
         "plt.show()"
       ],
       "metadata": {
@@ -159,7 +158,7 @@
         "# Now let's try max-pooling\n",
         "def maxpool(x_in):\n",
         "  x_out = np.zeros(( int(np.floor(x_in.shape[0]/2)), int(np.floor(x_in.shape[1]/2)) ))\n",
-        "  # TO DO -- write the maxpool routine\n",
+        "  # TODO -- write the maxpool routine\n",
         "  # Replace this line\n",
         "  x_out = x_out\n",
         "\n",
@@ -230,7 +229,7 @@
         "# Finally, let's try mean pooling\n",
         "def meanpool(x_in):\n",
         "  x_out = np.zeros(( int(np.floor(x_in.shape[0]/2)), int(np.floor(x_in.shape[1]/2)) ))\n",
-        "  # TO DO -- write the meanpool routine\n",
+        "  # TODO -- write the meanpool routine\n",
         "  # Replace this line\n",
         "  x_out = x_out\n",
         "\n",
@@ -301,7 +300,7 @@
       "cell_type": "code",
       "source": [
         "# Define 2 by 2 original patch\n",
-        "orig_2_2 = np.array([[2, 4], [4,8]])\n",
+        "orig_2_2 = np.array([[6, 8], [8,4]])\n",
         "print(orig_2_2)"
       ],
       "metadata": {
@@ -316,7 +315,7 @@
         "# Let's first use the duplication method\n",
         "def duplicate(x_in):\n",
         "  x_out = np.zeros(( x_in.shape[0]*2, x_in.shape[1]*2 ))\n",
-        "  # TO DO -- write the duplication routine\n",
+        "  # TODO -- write the duplication routine\n",
         "  # Replace this line\n",
         "  x_out = x_out\n",
         "\n",
@@ -345,11 +344,11 @@
     {
       "cell_type": "code",
       "source": [
-        "# Let's re-upsample, sub-sampled rick\n",
-        "data_duplicate = duplicate(data_subsample3);\n",
+        "# Let's re-upsample, downsampled rick\n",
+        "data_duplicate = duplicate(data_downsample3);\n",
         "\n",
         "plt.figure(figsize=(5,5))\n",
-        "plt.imshow(data_subsample3, cmap='gray')\n",
+        "plt.imshow(data_downsample3, cmap='gray')\n",
         "plt.show()\n",
         "\n",
         "plt.figure(figsize=(5,5))\n",
@@ -388,7 +387,7 @@
         "# The input x_high_res is the original high res image, from which you can deduce the position of the maximum index\n",
         "def max_unpool(x_in, x_high_res):\n",
         "  x_out = np.zeros(( x_in.shape[0]*2, x_in.shape[1]*2 ))\n",
-        "  # TO DO -- write the subsampling routine\n",
+        "  # TODO -- write the unpooling routine\n",
         "  # Replace this line\n",
         "  x_out = x_out\n",
         "\n",
@@ -417,7 +416,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Let's re-upsample, sub-sampled rick\n",
+        "# Let's re-upsample, down-sampled rick\n",
         "data_max_unpool= max_unpool(data_maxpool3,data_maxpool2);\n",
         "\n",
         "plt.figure(figsize=(5,5))\n",
@@ -460,7 +459,7 @@
         "  x_out = np.zeros(( x_in.shape[0]*2, x_in.shape[1]*2 ))\n",
         "  x_in_pad = np.zeros((x_in.shape[0]+1, x_in.shape[1]+1))\n",
         "  x_in_pad[0:x_in.shape[0],0:x_in.shape[1]] = x_in\n",
-        "  # TO DO -- write the duplication routine\n",
+        "  # TODO -- write the duplication routine\n",
         "  # Replace this line\n",
         "  x_out = x_out\n",
         "\n",
@@ -489,7 +488,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Let's re-upsample, sub-sampled rick\n",
+        "# Let's re-upsample, down-sampled rick\n",
         "data_bilinear = bilinear(data_meanpool3);\n",
         "\n",
         "plt.figure(figsize=(5,5))\n",

Notebooks/Chap10/10_5_Convolution_For_MNIST.ipynb

@@ -1,26 +1,10 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "authorship_tag": "ABX9TyNAcc98STMeyQgh9SbVHWG+",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
       "cell_type": "markdown",
       "metadata": {
-        "id": "view-in-github",
-        "colab_type": "text"
+        "colab_type": "text",
+        "id": "view-in-github"
       },
       "source": [
         "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/Notebooks/Chap10/10_5_Convolution_For_MNIST.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
@@ -28,6 +12,9 @@
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "t9vk9Elugvmi"
+      },
       "source": [
         "# **Notebook 10.5: Convolution for MNIST**\n",
         "\n",
@@ -35,16 +22,20 @@
         "\n",
         "The code is adapted from https://nextjournal.com/gkoehler/pytorch-mnist\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "\n",
+        "If you are using Google Colab, you can change your runtime to an instance with GPU support to speed up training, e.g. a T4 GPU. If you do this, the cell below should output ``device(type='cuda')``\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
-      ],
-      "metadata": {
-        "id": "t9vk9Elugvmi"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "YrXWAH7sUWvU"
+      },
+      "outputs": [],
       "source": [
         "import torch\n",
         "import torchvision\n",
@@ -52,23 +43,34 @@
         "import torch.nn.functional as F\n",
         "import torch.optim as optim\n",
         "import matplotlib.pyplot as plt\n",
-        "import random"
-      ],
-      "metadata": {
-        "id": "YrXWAH7sUWvU"
-      },
-      "execution_count": null,
-      "outputs": []
+        "import random\n",
+        "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
+        "device"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "wScBGXXFVadm"
+      },
+      "outputs": [],
       "source": [
         "# Run this once to load the train and test data straight into a dataloader class\n",
         "# that will provide the batches\n",
+        "\n",
+        "# (It may complain that some files are missing because the files seem to have been\n",
+        "# reorganized on the underlying website, but it still seems to work). If everything is working\n",
+        "# properly, then the whole notebook should run to the end without further problems\n",
+        "# even before you make changes.\n",
         "batch_size_train = 64\n",
         "batch_size_test = 1000\n",
+        "\n",
+        "# TODO Change this directory to point towards an existing directory (No change needed if using Google Colab)\n",
+        "myDir = '/files/'\n",
+        "\n",
         "train_loader = torch.utils.data.DataLoader(\n",
-        "  torchvision.datasets.MNIST('/files/', train=True, download=True,\n",
+        "  torchvision.datasets.MNIST(myDir, train=True, download=True,\n",
         "                             transform=torchvision.transforms.Compose([\n",
         "                               torchvision.transforms.ToTensor(),\n",
         "                               torchvision.transforms.Normalize(\n",
@@ -77,22 +79,22 @@
         "  batch_size=batch_size_train, shuffle=True)\n",
         "\n",
         "test_loader = torch.utils.data.DataLoader(\n",
-        "  torchvision.datasets.MNIST('/files/', train=False, download=True,\n",
+        "  torchvision.datasets.MNIST(myDir, train=False, download=True,\n",
         "                             transform=torchvision.transforms.Compose([\n",
         "                               torchvision.transforms.ToTensor(),\n",
         "                               torchvision.transforms.Normalize(\n",
         "                                 (0.1307,), (0.3081,))\n",
         "                             ])),\n",
         "  batch_size=batch_size_test, shuffle=True)"
-      ],
-      "metadata": {
-        "id": "wScBGXXFVadm"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "8bKADvLHbiV5"
+      },
+      "outputs": [],
       "source": [
         "# Let's draw some of the training data\n",
         "examples = enumerate(test_loader)\n",
@@ -107,24 +109,24 @@
         "  plt.xticks([])\n",
         "  plt.yticks([])\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "8bKADvLHbiV5"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Define the network.  This is a more typical way to define a network than the sequential structure.  We define a class for the network, and define the parameters in the constructor.  Then we use a function called forward to actually run the network.  It's easy to see how you might use residual connections in this format."
-      ],
       "metadata": {
         "id": "_sFvRDGrl4qe"
-      }
+      },
+      "source": [
+        "Define the network.  This is a more typical way to define a network than the sequential structure.  We define a class for the network, and define the parameters in the constructor.  Then we use a function called forward to actually run the network.  It's easy to see how you might use residual connections in this format."
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "EQkvw2KOPVl7"
+      },
+      "outputs": [],
       "source": [
         "from os import X_OK\n",
         "# TODO Change this class to implement\n",
@@ -165,52 +167,54 @@
         "\n",
         "\n",
         "\n"
-      ],
-      "metadata": {
-        "id": "EQkvw2KOPVl7"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "qWZtkCZcU_dg"
+      },
+      "outputs": [],
       "source": [
         "# He initialization of weights\n",
         "def weights_init(layer_in):\n",
         "  if isinstance(layer_in, nn.Linear):\n",
         "    nn.init.kaiming_uniform_(layer_in.weight)\n",
         "    layer_in.bias.data.fill_(0.0)"
-      ],
-      "metadata": {
-        "id": "qWZtkCZcU_dg"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "FslroPJJffrh"
+      },
+      "outputs": [],
       "source": [
         "# Create network\n",
-        "model = Net()\n",
+        "model = Net().to(device)\n",
         "# Initialize model weights\n",
         "model.apply(weights_init)\n",
         "# Define optimizer\n",
         "optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.5)"
-      ],
-      "metadata": {
-        "id": "FslroPJJffrh"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "xKQd9PzkQ766"
+      },
+      "outputs": [],
       "source": [
         "# Main training routine\n",
         "def train(epoch):\n",
         "  model.train()\n",
         "  # Get each\n",
         "  for batch_idx, (data, target) in enumerate(train_loader):\n",
+        "    data = data.to(device)\n",
+        "    target = target.to(device)\n",
         "    optimizer.zero_grad()\n",
         "    output = model(data)\n",
         "    loss = F.nll_loss(output, target)\n",
@@ -220,15 +224,15 @@
         "    if batch_idx % 10 == 0:\n",
         "      print('Train Epoch: {} [{}/{}]\\tLoss: {:.6f}'.format(\n",
         "        epoch, batch_idx * len(data), len(train_loader.dataset), loss.item()))"
-      ],
-      "metadata": {
-        "id": "xKQd9PzkQ766"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Byn-f7qWRLxX"
+      },
+      "outputs": [],
       "source": [
         "# Run on test data\n",
         "def test():\n",
@@ -237,6 +241,8 @@
         "  correct = 0\n",
         "  with torch.no_grad():\n",
         "    for data, target in test_loader:\n",
+        "      data = data.to(device)\n",
+        "      target = target.to(device)\n",
         "      output = model(data)\n",
         "      test_loss += F.nll_loss(output, target, size_average=False).item()\n",
         "      pred = output.data.max(1, keepdim=True)[1]\n",
@@ -245,15 +251,15 @@
         "  print('\\nTest set: Avg. loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\\n'.format(\n",
         "    test_loss, correct, len(test_loader.dataset),\n",
         "    100. * correct / len(test_loader.dataset)))"
-      ],
-      "metadata": {
-        "id": "Byn-f7qWRLxX"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "YgLaex1pfhqz"
+      },
+      "outputs": [],
       "source": [
         "# Get initial performance\n",
         "test()\n",
@@ -262,15 +268,15 @@
         "for epoch in range(1, n_epochs + 1):\n",
         "  train(epoch)\n",
         "  test()"
-      ],
-      "metadata": {
-        "id": "YgLaex1pfhqz"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "o7fRUAy9Se1B"
+      },
+      "outputs": [],
       "source": [
         "# Run network on data we got before and show predictions\n",
         "output = model(example_data)\n",
@@ -285,12 +291,23 @@
         "  plt.xticks([])\n",
         "  plt.yticks([])\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "o7fRUAy9Se1B"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     }
-  ]
-}
+  ],
+  "metadata": {
+    "colab": {
+      "authorship_tag": "ABX9TyORZF8xy4X1yf4oRhRq8Rtm",
+      "include_colab_link": true,
+      "provenance": []
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}

Notebooks/Chap11/11_1_Shattered_Gradients.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyMrF4rB2hTKq7XzLuYsURdL",
+      "authorship_tag": "ABX9TyP3VmRg51U+7NCfSYjRRrgv",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +33,7 @@
         "\n",
         "This notebook investigates the phenomenon of shattered gradients as discussed in section 11.1.1.  It replicates some of the experiments in [Balduzzi et al. (2017)](https://arxiv.org/abs/1702.08591).\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -65,7 +65,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# K is width, D is number of hidden units in each layer\n",
+        "# K is depth, D is number of hidden units in each layer\n",
         "def init_params(K, D):\n",
         "  # Set seed so we always get the same random numbers\n",
         "  np.random.seed(1)\n",
@@ -235,7 +235,7 @@
         "# Finite difference calculation\n",
         "dydx_fd = (y2-y1)/delta\n",
         "\n",
-        "print(\"Gradient calculation=%f, Finite difference gradient=%f\"%(dydx,dydx_fd))\n"
+        "print(\"Gradient calculation=%f, Finite difference gradient=%f\"%(dydx.squeeze(),dydx_fd.squeeze()))\n"
       ],
       "metadata": {
         "id": "KJpQPVd36Haq"
@@ -267,8 +267,8 @@
         "  fig,ax = plt.subplots()\n",
         "  ax.plot(np.squeeze(x_in), np.squeeze(dydx), 'b-')\n",
         "  ax.set_xlim(-2,2)\n",
-        "  ax.set_xlabel('Input, $x$')\n",
-        "  ax.set_ylabel('Gradient, $dy/dx$')\n",
+        "  ax.set_xlabel(r'Input, $x$')\n",
+        "  ax.set_ylabel(r'Gradient, $dy/dx$')\n",
         "  ax.set_title('No layers = %d'%(K))\n",
         "  plt.show()"
       ],
@@ -389,4 +389,4 @@
       "outputs": []
     }
   ]
-}
+}

Notebooks/Chap11/11_2_Residual_Networks.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyObut1y9atNUuowPT6dMY+I",
+      "authorship_tag": "ABX9TyNIY8tswL9e48d5D53aSmHO",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +33,7 @@
         "\n",
         "This notebook adapts the networks for MNIST1D to use residual connections.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
         "\n"
@@ -45,8 +45,8 @@
     {
       "cell_type": "code",
       "source": [
-        "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
-        "!git clone https://github.com/greydanus/mnist1d"
+        "# Run this if you're in a Colab to install MNIST 1D repository\n",
+        "!pip install git+https://github.com/greydanus/mnist1d"
       ],
       "metadata": {
         "id": "D5yLObtZCi9J"
@@ -144,10 +144,10 @@
         "  def count_params(self):\n",
         "    return sum([p.view(-1).shape[0] for p in self.parameters()])\n",
         "\n",
-        "# # TODO -- Add residual connections to this model\n",
-        "# # The order of operations should similar to figure 11.5b\n",
-        "# # linear1 first, ReLU+linear2 in first residual block, ReLU+linear3 in second residual block), linear4 at end\n",
-        "# # Replace this function\n",
+        "# TODO -- Add residual connections to this model\n",
+        "# The order of operations within each block should similar to figure 11.5b\n",
+        "# ie., linear1 first, ReLU+linear2 in first residual block, ReLU+linear3 in second residual block), linear4 at end\n",
+        "# Replace this function\n",
         "  def forward(self, x):\n",
         "    h1 = self.linear1(x).relu()\n",
         "    h2 = self.linear2(h1).relu()\n",

Notebooks/Chap11/11_3_Batch_Normalization.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyOoGS+lY+EhGthebSO4smpj",
+      "authorship_tag": "ABX9TyPx2mM2zTHmDJeKeiE1RymT",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +33,7 @@
         "\n",
         "This notebook investigates the use of batch normalization in residual networks.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
         "\n"
@@ -45,8 +45,8 @@
     {
       "cell_type": "code",
       "source": [
-        "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
-        "!git clone https://github.com/greydanus/mnist1d"
+        "# Run this if you're in a Colab to install MNIST 1D repository\n",
+        "!pip install git+https://github.com/greydanus/mnist1d"
       ],
       "metadata": {
         "id": "D5yLObtZCi9J"
@@ -205,7 +205,8 @@
         "    self.linear3 = nn.Linear(hidden_size, hidden_size)\n",
         "    self.linear4 = nn.Linear(hidden_size, hidden_size)\n",
         "    self.linear5 = nn.Linear(hidden_size, hidden_size)\n",
-        "    self.linear6 = nn.Linear(hidden_size, output_size)\n",
+        "    self.linear6 = nn.Linear(hidden_size, hidden_size)\n",
+        "    self.linear7 = nn.Linear(hidden_size, output_size)\n",
         "\n",
         "  def count_params(self):\n",
         "    return sum([p.view(-1).shape[0] for p in self.parameters()])\n",
@@ -220,11 +221,11 @@
         "    print_variance(\"After second residual connection\",res2)\n",
         "    res3 = res2 + self.linear4(res2.relu())\n",
         "    print_variance(\"After third residual connection\",res3)\n",
-        "    res4 = res3 + self.linear4(res3.relu())\n",
+        "    res4 = res3 + self.linear5(res3.relu())\n",
         "    print_variance(\"After fourth residual connection\",res4)\n",
-        "    res5 = res4 + self.linear4(res4.relu())\n",
+        "    res5 = res4 + self.linear6(res4.relu())\n",
         "    print_variance(\"After fifth residual connection\",res5)\n",
-        "    return self.linear6(res5)"
+        "    return self.linear7(res5)"
       ],
       "metadata": {
         "id": "FslroPJJffrh"
@@ -272,7 +273,8 @@
         "    self.linear3 = nn.Linear(hidden_size, hidden_size)\n",
         "    self.linear4 = nn.Linear(hidden_size, hidden_size)\n",
         "    self.linear5 = nn.Linear(hidden_size, hidden_size)\n",
-        "    self.linear6 = nn.Linear(hidden_size, output_size)\n",
+        "    self.linear6 = nn.Linear(hidden_size, hidden_size)\n",
+        "    self.linear7 = nn.Linear(hidden_size, output_size)\n",
         "\n",
         "  def count_params(self):\n",
         "    return sum([p.view(-1).shape[0] for p in self.parameters()])\n",
@@ -287,11 +289,11 @@
         "    print_variance(\"After second residual connection\",res2)\n",
         "    res3 = res2 + self.linear4(res2.relu())\n",
         "    print_variance(\"After third residual connection\",res3)\n",
-        "    res4 = res3 + self.linear4(res3.relu())\n",
+        "    res4 = res3 + self.linear5(res3.relu())\n",
         "    print_variance(\"After fourth residual connection\",res4)\n",
-        "    res5 = res4 + self.linear4(res4.relu())\n",
+        "    res5 = res4 + self.linear6(res4.relu())\n",
         "    print_variance(\"After fifth residual connection\",res5)\n",
-        "    return self.linear6(res5)"
+        "    return self.linear7(res5)"
       ],
       "metadata": {
         "id": "5JvMmaRITKGd"

Notebooks/Chap12/12_1_Self_Attention.ipynb

@@ -31,9 +31,9 @@
       "source": [
         "# **Notebook 12.1: Self Attention**\n",
         "\n",
-        "This notebook builds a self-attnetion mechanism from scratch, as discussed in section 12.2 of the book.\n",
+        "This notebook builds a self-attention mechanism from scratch, as discussed in section 12.2 of the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
         "\n"
@@ -153,7 +153,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "We'll need a softmax function (equation 12.5) -- here, it will take a list of arbirtrary numbers and return a list where the elements are non-negative and sum to one\n"
+        "We'll need a softmax function (equation 12.5) -- here, it will take a list of arbitrary numbers and return a list where the elements are non-negative and sum to one\n"
       ],
       "metadata": {
         "id": "Se7DK6PGPSUk"
@@ -364,7 +364,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "TODO -- Investigate whether the self-attention mechanism is covariant with respect to permulation.\n",
+        "TODO -- Investigate whether the self-attention mechanism is covariant with respect to permutation.\n",
         "If it is, when we permute the columns of the input matrix $\\mathbf{X}$, the columns of the output matrix $\\mathbf{X}'$ will also be permuted.\n"
       ],
       "metadata": {

Notebooks/Chap12/12_2_Multihead_Self_Attention.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyMSk8qTqDYqFnRJVZKlsue0",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -29,11 +28,11 @@
     {
       "cell_type": "markdown",
       "source": [
-        "# **Notebook 12.1: Multhead Self-Attention**\n",
+        "# **Notebook 12.2: Multihead Self-Attention**\n",
         "\n",
-        "This notebook builds a multihead self-attentionm mechanism as in figure 12.6\n",
+        "This notebook builds a multihead self-attention mechanism as in figure 12.6\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
         "\n"
@@ -147,9 +146,7 @@
         "  exp_values = np.exp(data_in) ;\n",
         "  # Sum over columns\n",
         "  denom = np.sum(exp_values, axis = 0);\n",
-        "  # Replicate denominator to N rows\n",
-        "  denom = np.matmul(np.ones((data_in.shape[0],1)), denom[np.newaxis,:])\n",
-        "  # Compute softmax\n",
+        "  # Compute softmax (numpy broadcasts denominator to all rows automatically)\n",
         "  softmax = exp_values / denom\n",
         "  # return the answer\n",
         "  return softmax"
@@ -209,4 +206,4 @@
       "outputs": []
     }
   ]
-}
+}

Notebooks/Chap12/12_3_Tokenization.ipynb

@@ -33,7 +33,7 @@
         "\n",
         "This notebook builds set of tokens from a text string as in figure 12.8 of the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "I adapted this code from *SOMEWHERE*.  If anyone recognizes it, can you let me know and I will give the proper attribution or rewrite if the license is not permissive.\n",
         "\n",

Notebooks/Chap12/12_4_Decoding_Strategies.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyNPrHfkLWjy3NfDHRhGG3IE",
+      "authorship_tag": "ABX9TyPsZjfqVeHYh95Hzt+hCIO7",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +33,7 @@
         "\n",
         "This practical investigates neural decoding from transformer models.  \n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -409,7 +409,7 @@
         "  print(\"Choosing from %d tokens\"%(thresh_index))\n",
         "  # TODO:  Find the probability value to threshold\n",
         "  # Replace this line:\n",
-        "  thresh_prob = sorted_probs_decreasing[thresh_index]\n",
+        "  thresh_prob = 0.5\n",
         "\n",
         "\n",
         "\n",

Notebooks/Chap13/13_1_Graph_Representation.ipynb

@@ -33,7 +33,7 @@
         "\n",
         "This notebook investigates representing graphs with matrices as illustrated in figure 13.4 from the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
         "\n"

Notebooks/Chap13/13_2_Graph_Classification.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyOMSGUFWT+YN0fwYHpMmHJM",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +32,7 @@
         "\n",
         "This notebook investigates representing graphs with matrices as illustrated in figure 13.4 from the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -57,7 +56,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Let's build a model that maps a chemical structure to a binary decision.  This model might be used to predict whether a chemical is liquid at room temparature or not.  We'll start by drawing the chemical structure."
+        "Let's build a model that maps a chemical structure to a binary decision.  This model might be used to predict whether a chemical is liquid at room temperature or not.  We'll start by drawing the chemical structure."
       ],
       "metadata": {
         "id": "UNleESc7k5uB"
@@ -99,7 +98,7 @@
         "\n",
         "# TODO -- Define node matrix\n",
         "# There will be 9 nodes and 118 possible chemical elements\n",
-        "# so we'll define a 9x118 matrix.  Each column represents one\n",
+        "# so we'll define a 118x9 matrix.  Each column represents one\n",
         "# node and is a one-hot vector (i.e. all zeros, except a single one at the\n",
         "# chemical number of the element).\n",
         "# Chemical numbers:  Hydrogen-->1, Carbon-->6,  Oxygen-->8\n",
@@ -191,7 +190,7 @@
       "source": [
         "# Let's test this network\n",
         "f = graph_neural_network(A,X, Omega0, beta0, Omega1, beta1, Omega2, beta2, omega3, beta3)\n",
-        "print(\"Your value is %3f: \"%(f[0,0]), \"True value of f: 0.498010\")"
+        "print(\"Your value is %3f: \"%(f[0,0]), \"True value of f: 0.310843\")"
       ],
       "metadata": {
         "id": "X7gYgOu6uIAt"
@@ -221,7 +220,7 @@
         "X_permuted = np.copy(X)\n",
         "\n",
         "f = graph_neural_network(A_permuted,X_permuted, Omega0, beta0, Omega1, beta1, Omega2, beta2, omega3, beta3)\n",
-        "print(\"Your value is %3f: \"%(f[0,0]), \"True value of f: 0.498010\")"
+        "print(\"Your value is %3f: \"%(f[0,0]), \"True value of f: 0.310843\")"
       ],
       "metadata": {
         "id": "F0zc3U_UuR5K"

Notebooks/Chap13/13_3_Neighborhood_Sampling.ipynb

@@ -33,7 +33,7 @@
         "\n",
         "This notebook investigates neighborhood sampling of graphs as in figure 13.10 from the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -268,7 +268,7 @@
       "source": [
         "# TODO Find the nodes in hidden layer 1 that connect to the nodes in hidden layer 2\n",
         "# using the adjacency matrix.  Then sample n_sample of these nodes randomly without\n",
-        "# replacement.  Make sure not to sample nodes that were already included in hidden layer 2 our the ouput layer.\n",
+        "# replacement.  Make sure not to sample nodes that were already included in hidden layer 2 our the output layer.\n",
         "# The nodes at hidden layer 1 are the union of these nodes and the nodes in hidden layer 2\n",
         "\n",
         "# Replace this line:\n",
@@ -311,4 +311,4 @@
       }
     }
   ]
-}
+}

Notebooks/Chap13/13_4_Graph_Attention_Networks.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyOdSkjfQnSZXnffGsZVM7r5",
+      "authorship_tag": "ABX9TyO/wJ4N9w01f04mmrs/ZSHY",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +33,7 @@
         "\n",
         "This notebook builds a graph attention mechanism from scratch, as discussed in section 13.8.6 of the book and illustrated in figure 13.12c\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
         "\n"
@@ -109,7 +109,7 @@
         "# Choose random values for the parameters\n",
         "omega = np.random.normal(size=(D,D))\n",
         "beta = np.random.normal(size=(D,1))\n",
-        "phi = np.random.normal(size=(1,2*D))"
+        "phi = np.random.normal(size=(2*D,1))"
       ],
       "metadata": {
         "id": "79TSK7oLMobe"
@@ -185,10 +185,10 @@
         "np.set_printoptions(precision=3)\n",
         "output = graph_attention(X, omega, beta, phi, A);\n",
         "print(\"Correct answer is:\")\n",
-        "print(\"[[1.796 1.346 0.569 1.703 1.298 1.224 1.24  1.234]\")\n",
-        "print(\" [0.768 0.672 0.    0.529 3.841 4.749 5.376 4.761]\")\n",
-        "print(\" [0.305 0.129 0.    0.341 0.785 1.014 1.113 1.024]\")\n",
-        "print(\" [0.    0.    0.    0.    0.35  0.864 1.098 0.871]]]\")\n",
+        "print(\"[[0.    0.028 0.37  0.    0.97  0.    0.    0.698]\")\n",
+        "print(\" [0.    0.    0.    0.    1.184 0.    2.654 0.  ]\")\n",
+        "print(\" [1.13  0.564 0.    1.298 0.268 0.    0.    0.779]\")\n",
+        "print(\" [0.825 0.    0.    1.175 0.    0.    0.    0.  ]]]\")\n",
         "\n",
         "\n",
         "print(\"Your answer is:\")\n",
@@ -210,4 +210,4 @@
       }
     }
   ]
-}
+}

Notebooks/Chap15/15_1_GAN_Toy_Example.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyM0StKV3FIZ3MZqfflqC0Rv",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -31,9 +30,9 @@
       "source": [
         "# **Notebook 15.1: GAN Toy example**\n",
         "\n",
-        "This notebook investigates the GAN toy example as illustred in figure 15.1 in the book.\n",
+        "This notebook investigates the GAN toy example as illustrated in figure 15.1 in the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -101,7 +100,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Now, we define our disriminator.  This is a simple logistic regression model (1D linear model passed through sigmoid) that returns the probability that the data is real"
+        "Now, we define our discriminator.  This is a simple logistic regression model (1D linear model passed through sigmoid) that returns the probability that the data is real"
       ],
       "metadata": {
         "id": "Xrzd8aehYAYR"
@@ -339,7 +338,7 @@
         "    print(\"Initial generator loss = \", compute_generator_loss(z, theta, phi0, phi1))\n",
         "    for iter in range(n_iter):\n",
         "      # Get gradient\n",
-        "      dl_dtheta = compute_generator_gradient(x_real, x_syn, phi0, phi1)\n",
+        "      dl_dtheta = compute_generator_gradient(z, theta, phi0, phi1)\n",
         "      # Take a gradient step (uphill, since we are trying to make synthesized data less well classified by discriminator)\n",
         "      theta = theta + alpha * dl_dtheta ;\n",
         "\n",
@@ -387,7 +386,7 @@
         "print(\"Final parameters (phi0,phi1)\", phi0, phi1)\n",
         "for c_gan_iter in range(5):\n",
         "\n",
-        "  # Run generator to product syntehsized data\n",
+        "  # Run generator to product synthesized data\n",
         "  x_syn = generator(z, theta)\n",
         "  draw_data_model(x_real, x_syn, phi0, phi1)\n",
         "\n",

Notebooks/Chap15/15_2_Wasserstein_Distance.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyNyLnpoXgKN+RGCuTUszCAZ",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -29,11 +28,11 @@
     {
       "cell_type": "markdown",
       "source": [
-        "# **Notebook 15.2: Wassserstein Distance**\n",
+        "# **Notebook 15.2: Wasserstein Distance**\n",
         "\n",
-        "This notebook investigates the GAN toy example as illustred in figure 15.1 in the book.\n",
+        "This notebook investigates the GAN toy example as illustrated in figure 15.1 in the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -87,6 +86,7 @@
       "cell_type": "code",
       "source": [
         "# TODO Define the distance matrix from figure 15.8d\n",
+        "# The index should be normalized before being used in the distance calculation.\n",
         "# Replace this line\n",
         "dist_mat = np.zeros((10,10))\n",
         "\n",
@@ -129,7 +129,7 @@
     {
       "cell_type": "code",
       "source": [
-        "draw_2D_heatmap(dist_mat,'Distance $|i-j|$', my_colormap)"
+        "draw_2D_heatmap(dist_mat,r'Distance $|i-j|$', my_colormap)"
       ],
       "metadata": {
         "id": "G0HFPBXyHT6V"
@@ -153,9 +153,9 @@
       "cell_type": "code",
       "source": [
         "# TODO:  Now construct the matrix A that has the initial distribution constraints\n",
-        "# so that Ap=b where p is the transport plan P vectorized rows first so p = np.flatten(P)\n",
+        "# so that A @ TPFlat=b where TPFlat is the transport plan TP vectorized rows first so TPFlat = np.flatten(TP)\n",
         "# Replace this line:\n",
-        "A = np.zeros((20,100))\n"
+        "A = np.zeros((20,100))"
       ],
       "metadata": {
         "id": "7KrybL96IuNW"
@@ -197,8 +197,8 @@
     {
       "cell_type": "code",
       "source": [
-        "P = np.array(opt.x).reshape(10,10)\n",
-        "draw_2D_heatmap(P,'Transport plan $\\mathbf{P}$', my_colormap)"
+        "TP = np.array(opt.x).reshape(10,10)\n",
+        "draw_2D_heatmap(TP,r'Transport plan $\\mathbf{P}$', my_colormap)"
       ],
       "metadata": {
         "id": "nZGfkrbRV_D0"
@@ -218,8 +218,9 @@
     {
       "cell_type": "code",
       "source": [
-        "was = np.sum(P * dist_mat)\n",
-        "print(\"Wasserstein distance = \", was)"
+        "was = np.sum(TP * dist_mat)\n",
+        "print(\"Your Wasserstein distance = \", was)\n",
+        "print(\"Correct answer =  0.15148578811369506\")"
       ],
       "metadata": {
         "id": "yiQ_8j-Raq3c"
@@ -243,4 +244,4 @@
       }
     }
   ]
-}
+}

Notebooks/Chap16/16_1_1D_Normalizing_Flows.ipynb

@@ -33,7 +33,7 @@
         "\n",
         "This notebook investigates a 1D normalizing flows example similar to that illustrated in figures 16.1 to 16.3 in the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],

Notebooks/Chap16/16_2_Autoregressive_Flows.ipynb

@@ -33,7 +33,7 @@
         "\n",
         "This notebook investigates a 1D normalizing flows example similar to that illustrated in figure 16.7 in the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -65,7 +65,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# First let's make the 1D piecewise linear mapping as illustated in figure 16.5\n",
+        "# First let's make the 1D piecewise linear mapping as illustrated in figure 16.5\n",
         "def g(h, phi):\n",
         "  # TODO -- write this function (equation 16.12)\n",
         "  # Note: If you have the first printing of the book, there is a mistake in equation 16.12\n",
@@ -156,7 +156,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Now let's define an autogressive flow.  Let's switch to looking at figure 16.7.# We'll assume that our piecewise function will use five parameters phi1,phi2,phi3,phi4,phi5"
+        "Now let's define an autoregressive flow.  Let's switch to looking at figure 16.7.# We'll assume that our piecewise function will use five parameters phi1,phi2,phi3,phi4,phi5"
       ],
       "metadata": {
         "id": "t8XPxipfd7hz"
@@ -175,7 +175,7 @@
         "  x = x/ np.sum(x) ;\n",
         "  return x\n",
         "\n",
-        "# Return value of phi that doesn't depend on any of the iputs\n",
+        "# Return value of phi that doesn't depend on any of the inputs\n",
         "def get_phi():\n",
         "  return np.array([0.2, 0.1, 0.4, 0.05, 0.25])\n",
         "\n",

Notebooks/Chap16/16_3_Contraction_Mappings.ipynb

@@ -1,73 +1,64 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "authorship_tag": "ABX9TyNeCWINUqqUGKMcxsqPFTAh",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
+      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
-        "id": "view-in-github",
-        "colab_type": "text"
+        "colab_type": "text",
+        "id": "view-in-github"
       },
       "source": [
         "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/Notebooks/Chap16/16_3_Contraction_Mappings.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
       ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
+      "metadata": {
+        "id": "t9vk9Elugvmi"
+      },
       "source": [
         "# **Notebook 16.3: Contraction mappings**\n",
         "\n",
         "This notebook investigates a 1D normalizing flows example similar to that illustrated in figure 16.9 in the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
-      ],
-      "metadata": {
-        "id": "t9vk9Elugvmi"
-      }
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "import numpy as np\n",
-        "import matplotlib.pyplot as plt"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "OLComQyvCIJ7"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "import numpy as np\n",
+        "import matplotlib.pyplot as plt"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "4Pfz2KSghdVI"
+      },
+      "outputs": [],
       "source": [
         "# Define a function that is a contraction mapping\n",
         "def f(z):\n",
         "    return 0.3 + 0.5 *z + 0.02 * np.sin(z*15)"
-      ],
-      "metadata": {
-        "id": "4Pfz2KSghdVI"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "zEwCbIx0hpAI"
+      },
+      "outputs": [],
       "source": [
         "def draw_function(f, fixed_point=None):\n",
         "  z = np.arange(0,1,0.01)\n",
@@ -84,35 +75,36 @@
         "  ax.set_xlabel('Input, $z$')\n",
         "  ax.set_ylabel('Output, f$[z]$')\n",
         "  plt.show()"
-      ],
-      "metadata": {
-        "id": "zEwCbIx0hpAI"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "draw_function(f)"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "k4e5Yu0fl8bz"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "draw_function(f)"
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Now let's find where $\\mbox{f}[z]=z$ using fixed point iteration"
-      ],
       "metadata": {
         "id": "DfgKrpCAjnol"
-      }
+      },
+      "source": [
+        "Now let's find where $\\text{f}[z]=z$ using fixed point iteration"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "bAOBvZT-j3lv"
+      },
+      "outputs": [],
       "source": [
         "# Takes a function f and a starting point z\n",
         "def fixed_point_iteration(f, z0):\n",
@@ -125,115 +117,117 @@
         "\n",
         "\n",
         "  return z_out"
-      ],
-      "metadata": {
-        "id": "bAOBvZT-j3lv"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Now let's test that and plot the solution"
-      ],
       "metadata": {
         "id": "CAS0lgIomAa0"
-      }
+      },
+      "source": [
+        "Now let's test that and plot the solution"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "EYQZJdNPk8Lg"
+      },
+      "outputs": [],
       "source": [
         "# Now let's test that\n",
         "z = fixed_point_iteration(f, 0.2)\n",
         "draw_function(f, z)"
-      ],
-      "metadata": {
-        "id": "EYQZJdNPk8Lg"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "4DipPiqVlnwJ"
+      },
+      "outputs": [],
       "source": [
         "# Let's define another function\n",
         "def f2(z):\n",
         "    return 0.7 + -0.6 *z + 0.03 * np.sin(z*15)\n",
         "draw_function(f2)"
-      ],
-      "metadata": {
-        "id": "4DipPiqVlnwJ"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "tYOdbWcomdEE"
+      },
+      "outputs": [],
       "source": [
         "# Now let's test that\n",
         "# TODO Before running this code, predict what you think will happen\n",
         "z = fixed_point_iteration(f2, 0.9)\n",
         "draw_function(f2, z)"
-      ],
-      "metadata": {
-        "id": "tYOdbWcomdEE"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Mni37RUpmrIu"
+      },
+      "outputs": [],
       "source": [
         "# Let's define another function\n",
         "# Define a function that is a contraction mapping\n",
         "def f3(z):\n",
         "    return -0.2 + 1.5 *z + 0.1 * np.sin(z*15)\n",
         "draw_function(f3)"
-      ],
-      "metadata": {
-        "id": "Mni37RUpmrIu"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "agt5mfJrnM1O"
+      },
+      "outputs": [],
       "source": [
         "# Now let's test that\n",
         "# TODO Before running this code, predict what you think will happen\n",
         "z = fixed_point_iteration(f3, 0.7)\n",
         "draw_function(f3, z)"
-      ],
-      "metadata": {
-        "id": "agt5mfJrnM1O"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Finally, let's invert a problem of the form $y = z+ f[z]$  for a given value of $y$. What is the $z$ that maps to it?"
-      ],
       "metadata": {
         "id": "n6GI46-ZoQz6"
-      }
+      },
+      "source": [
+        "Finally, let's invert a problem of the form $y = z+ f[z]$  for a given value of $y$. What is the $z$ that maps to it?"
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "def f4(z):\n",
-        "   return -0.3 + 0.5 *z + 0.02 * np.sin(z*15)"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "dy6r3jr9rjPf"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "def f4(z):\n",
+        "   return -0.3 + 0.5 *z + 0.02 * np.sin(z*15)"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "GMX64Iz0nl-B"
+      },
+      "outputs": [],
       "source": [
         "def fixed_point_iteration_z_plus_f(f, y, z0):\n",
         "  # TODO -- write this function\n",
@@ -241,15 +235,15 @@
         "  z_out = 1\n",
         "\n",
         "  return z_out"
-      ],
-      "metadata": {
-        "id": "GMX64Iz0nl-B"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "uXxKHad5qT8Y"
+      },
+      "outputs": [],
       "source": [
         "def draw_function2(f, y, fixed_point=None):\n",
         "  z = np.arange(0,1,0.01)\n",
@@ -267,15 +261,15 @@
         "  ax.set_xlabel('Input, $z$')\n",
         "  ax.set_ylabel('Output, z+f$[z]$')\n",
         "  plt.show()"
-      ],
-      "metadata": {
-        "id": "uXxKHad5qT8Y"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "mNEBXC3Aqd_1"
+      },
+      "outputs": [],
       "source": [
         "# Test this out and draw\n",
         "y = 0.8\n",
@@ -283,12 +277,23 @@
         "draw_function2(f4,y,z)\n",
         "# If you have done this correctly, the red dot should be\n",
         "# where the cyan curve has a y value of 0.8"
-      ],
-      "metadata": {
-        "id": "mNEBXC3Aqd_1"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     }
-  ]
-}
+  ],
+  "metadata": {
+    "colab": {
+      "authorship_tag": "ABX9TyNeCWINUqqUGKMcxsqPFTAh",
+      "include_colab_link": true,
+      "provenance": []
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}

Notebooks/Chap17/17_1_Latent_Variable_Models.ipynb

@@ -1,20 +1,4 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "authorship_tag": "ABX9TyMBYNsjj1iTgHUYhAXqUYJd",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
       "cell_type": "markdown",
@@ -28,80 +12,85 @@
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "t9vk9Elugvmi"
+      },
       "source": [
         "# **Notebook 17.1: Latent variable models**\n",
         "\n",
         "This notebook investigates a non-linear latent variable model similar to that in figures 17.2 and 17.3 of the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
-      ],
-      "metadata": {
-        "id": "t9vk9Elugvmi"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "OLComQyvCIJ7"
+      },
+      "outputs": [],
       "source": [
         "import numpy as np\n",
         "import matplotlib.pyplot as plt\n",
         "import scipy\n",
         "from matplotlib.colors import ListedColormap\n",
         "from matplotlib import cm"
-      ],
-      "metadata": {
-        "id": "OLComQyvCIJ7"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "IyVn-Gi-p7wf"
+      },
       "source": [
         "We'll assume that our base distribution over the latent variables is a 1D standard normal so that\n",
         "\n",
         "\\begin{equation}\n",
-        "Pr(z) = \\mbox{Norm}_{z}[0,1]\n",
+        "Pr(z) = \\text{Norm}_{z}[0,1]\n",
         "\\end{equation}\n",
         "\n",
-        "As in figure 17.2, we'll assume that the output is two dimensional, we we need to define a function that maps from the 1D latent variable to two dimensions.  Usually, we would use a neural network, but in this case, we'll just define an arbitrary relationship.\n",
+        "As in figure 17.2, we'll assume that the output is two dimensional, we need to define a function that maps from the 1D latent variable to two dimensions.  Usually, we would use a neural network, but in this case, we'll just define an arbitrary relationship.\n",
         "\n",
-        "\\begin{eqnarray}\n",
+        "\\begin{align}\n",
         "x_{1} &=& 0.5\\cdot\\exp\\Bigl[\\sin\\bigl[2+ 3.675 z \\bigr]\\Bigr]\\\\\n",
         "x_{2} &=& \\sin\\bigl[2+ 2.85 z \\bigr]\n",
-        "\\end{eqnarray}"
-      ],
-      "metadata": {
-        "id": "IyVn-Gi-p7wf"
-      }
+        "\\end{align}"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ZIfQwhd-AV6L"
+      },
+      "outputs": [],
       "source": [
         "# The function that maps z to x1 and x2\n",
         "def f(z):\n",
         "  x_1 = np.exp(np.sin(2+z*3.675)) * 0.5\n",
         "  x_2 = np.cos(2+z*2.85)\n",
         "  return x_1, x_2"
-      ],
-      "metadata": {
-        "id": "ZIfQwhd-AV6L"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Let's plot the 3D relation between the two observed variables $x_{1}$ and $x_{2}$ and the latent variables $z$ as in figure 17.2 of the book.  We'll use the opacity to represent the prior probability $Pr(z)$."
-      ],
       "metadata": {
         "id": "KB9FU34onW1j"
-      }
+      },
+      "source": [
+        "Let's plot the 3D relation between the two observed variables $x_{1}$ and $x_{2}$ and the latent variables $z$ as in figure 17.2 of the book.  We'll use the opacity to represent the prior probability $Pr(z)$."
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "lW08xqAgnP4q"
+      },
+      "outputs": [],
       "source": [
         "def draw_3d_projection(z,pr_z, x1,x2):\n",
         "  alpha = pr_z / np.max(pr_z)\n",
@@ -118,28 +107,28 @@
         "  ax.set_zlim(-1,1)\n",
         "  ax.set_box_aspect((3,1,1))\n",
         "  plt.show()"
-      ],
-      "metadata": {
-        "id": "lW08xqAgnP4q"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "9DUTauMi6tPk"
+      },
+      "outputs": [],
       "source": [
         "# Compute the prior\n",
         "def get_prior(z):\n",
         "  return scipy.stats.multivariate_normal.pdf(z)"
-      ],
-      "metadata": {
-        "id": "9DUTauMi6tPk"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "PAzHq461VqvF"
+      },
+      "outputs": [],
       "source": [
         "# Define the latent variable values\n",
         "z = np.arange(-3.0,3.0,0.01)\n",
@@ -149,40 +138,40 @@
         "x1,x2 = f(z)\n",
         "# Plot the function\n",
         "draw_3d_projection(z,pr_z, x1,x2)"
-      ],
-      "metadata": {
-        "id": "PAzHq461VqvF"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "The likelihood is defined as:\n",
-        "\\begin{eqnarray}\n",
-        " Pr(x_1,x_2|z) &=&  \\mbox{Norm}_{[x_1,x_2]}\\Bigl[\\mathbf{f}[z],\\sigma^{2}\\mathbf{I}\\Bigr]\n",
-        "\\end{eqnarray}\n",
-        "\n",
-        "so we will also need to define the noise level $\\sigma^2$"
-      ],
       "metadata": {
         "id": "sQg2gKR5zMrF"
-      }
+      },
+      "source": [
+        "The likelihood is defined as:\n",
+        "\\begin{align}\n",
+        " Pr(x_1,x_2|z) &=&  \\text{Norm}_{[x_1,x_2]}\\Bigl[\\mathbf{f}[z],\\sigma^{2}\\mathbf{I}\\Bigr]\n",
+        "\\end{align}\n",
+        "\n",
+        "so we will also need to define the noise level $\\sigma^2$"
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "sigma_sq = 0.04"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "In_Vg4_0nva3"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "sigma_sq = 0.04"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "6P6d-AgAqxXZ"
+      },
+      "outputs": [],
       "source": [
         "# Draws a heatmap to represent a probability distribution, possibly with samples overlaed\n",
         "def plot_heatmap(x1_mesh,x2_mesh,y_mesh, x1_samples=None, x2_samples=None, title=None):\n",
@@ -207,15 +196,15 @@
         "  ax.set_xlabel('$x_1$'); ax.set_ylabel('$x_2$')\n",
         "  plt.show()\n",
         "\n"
-      ],
-      "metadata": {
-        "id": "6P6d-AgAqxXZ"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "diYKb7_ZgjlJ"
+      },
+      "outputs": [],
       "source": [
         "# Returns the likelihood\n",
         "def get_likelihood(x1_mesh, x2_mesh, z_val):\n",
@@ -226,24 +215,24 @@
         "  mn = scipy.stats.multivariate_normal([x1, x2], [[sigma_sq, 0], [0, sigma_sq]])\n",
         "  pr_x1_x2_given_z_val = mn.pdf(np.dstack((x1_mesh, x2_mesh)))\n",
         "  return pr_x1_x2_given_z_val"
-      ],
-      "metadata": {
-        "id": "diYKb7_ZgjlJ"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Now let's plot the likelihood $Pr(x_1,x_2|z)$ as in fig 17.3b in the book."
-      ],
       "metadata": {
         "id": "0X4NwixzqxtZ"
-      }
+      },
+      "source": [
+        "Now let's plot the likelihood $Pr(x_1,x_2|z)$ as in fig 17.3b in the book."
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "hWfqK-Oz5_DT"
+      },
+      "outputs": [],
       "source": [
         "# Choose some z value\n",
         "z_val = 1.8\n",
@@ -253,64 +242,64 @@
         "pr_x1_x2_given_z_val = get_likelihood(x1_mesh,x2_mesh, z_val)\n",
         "\n",
         "# Plot the result\n",
-        "plot_heatmap(x1_mesh, x2_mesh, pr_x1_x2_given_z_val, title=\"Conditional distribution $Pr(x1,x2|z)$\")\n",
+        "plot_heatmap(x1_mesh, x2_mesh, pr_x1_x2_given_z_val, title=\"Conditional distribution $Pr(x_1,x_2|z)$\")\n",
         "\n",
         "# TODO -- Experiment with different values of z and make sure that you understand the what is happening."
-      ],
-      "metadata": {
-        "id": "hWfqK-Oz5_DT"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "25xqXnmFo-PH"
+      },
       "source": [
         "The data density is found by marginalizing over the latent variables $z$:\n",
         "\n",
-        "\\begin{eqnarray}\n",
+        "\\begin{align}\n",
         " Pr(x_1,x_2) &=& \\int Pr(x_1,x_2, z) dz \\nonumber \\\\\n",
         " &=& \\int Pr(x_1,x_2 | z) \\cdot Pr(z)dz\\nonumber \\\\\n",
-        " &=& \\int \\mbox{Norm}_{[x_1,x_2]}\\Bigl[\\mathbf{f}[z],\\sigma^{2}\\mathbf{I}\\Bigr]\\cdot \\mbox{Norm}_{z}\\left[\\mathbf{0},\\mathbf{I}\\right]dz.\n",
-        "\\end{eqnarray}"
-      ],
-      "metadata": {
-        "id": "25xqXnmFo-PH"
-      }
+        " &=& \\int \\text{Norm}_{[x_1,x_2]}\\Bigl[\\mathbf{f}[z],\\sigma^{2}\\mathbf{I}\\Bigr]\\cdot \\text{Norm}_{z}\\left[\\mathbf{0},\\mathbf{I}\\right]dz.\n",
+        "\\end{align}"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "H0Ijce9VzeCO"
+      },
+      "outputs": [],
       "source": [
         "# TODO Compute the data density\n",
         "# We can't integrate this function in closed form\n",
         "# So let's approximate it as a sum over the z values (z = np.arange(-3,3,0.01))\n",
         "# You will need the functions get_likelihood() and get_prior()\n",
-        "# To make this a valid probability distribution, you need to divide\n",
+        "# To make this a valid probability distribution, you need to multiply\n",
         "# By the z-increment (0.01)\n",
         "# Replace this line\n",
         "pr_x1_x2 = np.zeros_like(x1_mesh)\n",
         "\n",
         "\n",
         "# Plot the result\n",
-        "plot_heatmap(x1_mesh, x2_mesh, pr_x1_x2, title=\"Data density $Pr(x1,x2)$\")\n"
-      ],
-      "metadata": {
-        "id": "H0Ijce9VzeCO"
-      },
-      "execution_count": null,
-      "outputs": []
+        "plot_heatmap(x1_mesh, x2_mesh, pr_x1_x2, title=\"Data density $Pr(x_1,x_2)$\")\n"
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Now let's draw some samples from the model"
-      ],
       "metadata": {
         "id": "W264N7By_h9y"
-      }
+      },
+      "source": [
+        "Now let's draw some samples from the model"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Li3mK_I48k0k"
+      },
+      "outputs": [],
       "source": [
         "def draw_samples(n_sample):\n",
         "  # TODO Write this routine to draw n_sample samples from the model\n",
@@ -320,37 +309,37 @@
         "  x1_samples=0; x2_samples = 0;\n",
         "\n",
         "  return x1_samples, x2_samples"
-      ],
-      "metadata": {
-        "id": "Li3mK_I48k0k"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Let's plot those samples on top of the heat map."
-      ],
       "metadata": {
         "id": "D7N7oqLe-eJO"
-      }
+      },
+      "source": [
+        "Let's plot those samples on top of the heat map."
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "x1_samples, x2_samples = draw_samples(500)\n",
-        "# Plot the result\n",
-        "plot_heatmap(x1_mesh, x2_mesh, pr_x1_x2, x1_samples, x2_samples, title=\"Data density $Pr(x1,x2)$\")\n"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "XRmWv99B-BWO"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "x1_samples, x2_samples = draw_samples(500)\n",
+        "# Plot the result\n",
+        "plot_heatmap(x1_mesh, x2_mesh, pr_x1_x2, x1_samples, x2_samples, title=\"Data density $Pr(x_1,x_2)$\")\n"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "PwOjzPD5_1OF"
+      },
+      "outputs": [],
       "source": [
         "# Return the posterior distribution\n",
         "def get_posterior(x1,x2):\n",
@@ -364,15 +353,15 @@
         "\n",
         "\n",
         "  return z, pr_z_given_x1_x2"
-      ],
-      "metadata": {
-        "id": "PwOjzPD5_1OF"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "PKFUY42K-Tp7"
+      },
+      "outputs": [],
       "source": [
         "x1 = 0.9; x2 = -0.9\n",
         "z, pr_z_given_x1_x2 = get_posterior(x1,x2)\n",
@@ -385,12 +374,22 @@
         "ax.set_xlim([-3,3])\n",
         "ax.set_ylim([0,1.5 * np.max(pr_z_given_x1_x2)])\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "PKFUY42K-Tp7"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     }
-  ]
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
 }

Notebooks/Chap17/17_2_Reparameterization_Trick.ipynb

@@ -1,20 +1,4 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "authorship_tag": "ABX9TyOxO2/0DTH4n4zhC97qbagY",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
       "cell_type": "markdown",
@@ -28,38 +12,41 @@
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "t9vk9Elugvmi"
+      },
       "source": [
         "# **Notebook 17.2: Reparameterization trick**\n",
         "\n",
         "This notebook investigates the reparameterization trick as described in section 17.7 of the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
-      ],
-      "metadata": {
-        "id": "t9vk9Elugvmi"
-      }
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "import numpy as np\n",
-        "import matplotlib.pyplot as plt"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "OLComQyvCIJ7"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "import numpy as np\n",
+        "import matplotlib.pyplot as plt"
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "paLz5RukZP1J"
+      },
       "source": [
-        "The reparameterization trick computes the derivative of an expectation of a function $\\mbox{f}[x]$:\n",
+        "The reparameterization trick computes the derivative of an expectation of a function $\\text{f}[x]$:\n",
         "\n",
         "\\begin{equation}\n",
-        "\\frac{\\partial}{\\partial \\boldsymbol\\phi} \\mathbb{E}_{Pr(x|\\boldsymbol\\phi)}\\bigl[\\mbox{f}[x]\\bigr],\n",
+        "\\frac{\\partial}{\\partial \\boldsymbol\\phi} \\mathbb{E}_{Pr(x|\\boldsymbol\\phi)}\\bigl[\\text{f}[x]\\bigr],\n",
         "\\end{equation}\n",
         "\n",
         "with respect to the parameters $\\boldsymbol\\phi$ of the distribution $Pr(x|\\boldsymbol\\phi)$ that the expectation is over.\n",
@@ -67,23 +54,25 @@
         "Let's consider a simple concrete example, where:\n",
         "\n",
         "\\begin{equation}\n",
-        "Pr(x|\\phi) = \\mbox{Norm}_{x}\\Bigl[\\mu, \\sigma^2\\Bigr]=\\mbox{Norm}_{x}\\Bigl[\\phi^3,(\\exp[\\phi])^2\\Bigr]\n",
+        "Pr(x|\\phi) = \\text{Norm}_{x}\\Bigl[\\mu, \\sigma^2\\Bigr]=\\text{Norm}_{x}\\Bigl[\\phi^3,(\\exp[\\phi])^2\\Bigr]\n",
         "\\end{equation}\n",
         "\n",
         "and\n",
         "\n",
         "\\begin{equation}\n",
-        "\\mbox{f}[x] = x^2+\\sin[x]\n",
+        "\\text{f}[x] = x^2+\\sin[x]\n",
         "\\end{equation}"
-      ],
-      "metadata": {
-        "id": "paLz5RukZP1J"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "FdEbMnDBY0i9"
+      },
+      "outputs": [],
       "source": [
-        "# Let's approximate this expecctation for a particular value of phi\n",
+        "# Let's approximate this expectation for a particular value of phi\n",
         "def compute_expectation(phi, n_samples):\n",
         "  # TODO complete this function\n",
         "  # 1. Compute the mean of the normal distribution, mu\n",
@@ -96,15 +85,15 @@
         "\n",
         "\n",
         "  return expected_f_given_phi"
-      ],
-      "metadata": {
-        "id": "FdEbMnDBY0i9"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "FTh7LJ0llNJZ"
+      },
+      "outputs": [],
       "source": [
         "# Set the seed so the random numbers are all the same\n",
         "np.random.seed(0)\n",
@@ -119,24 +108,24 @@
         "n_samples = 10000000\n",
         "expected_f_given_phi2 = compute_expectation(phi2, n_samples)\n",
         "print(\"Your value: \", expected_f_given_phi2, \", True value:  0.8176793102849222\")"
-      ],
-      "metadata": {
-        "id": "FTh7LJ0llNJZ"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Le't plot this expectation as a function of phi"
-      ],
       "metadata": {
         "id": "r5Hl2QkimWx9"
-      }
+      },
+      "source": [
+        "Le't plot this expectation as a function of phi"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "05XxVLJxmkER"
+      },
+      "outputs": [],
       "source": [
         "phi_vals = np.arange(-1.5,1.5, 0.05)\n",
         "expected_vals = np.zeros_like(phi_vals)\n",
@@ -146,18 +135,16 @@
         "\n",
         "fig,ax = plt.subplots()\n",
         "ax.plot(phi_vals, expected_vals,'r-')\n",
-        "ax.set_xlabel('Parameter $\\phi$')\n",
-        "ax.set_ylabel('$\\mathbb{E}_{Pr(x|\\phi)}[f[x]]$')\n",
+        "ax.set_xlabel(r'Parameter $\\phi$')\n",
+        "ax.set_ylabel(r'$\\mathbb{E}_{Pr(x|\\phi)}[f[x]]$')\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "05XxVLJxmkER"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "zTCykVeWqj_O"
+      },
       "source": [
         "It's this curve that we want to find the derivative of (so for example, we could run gradient descent and find the minimum.\n",
         "\n",
@@ -166,28 +153,30 @@
         "The answer is the reparameterization trick.  We note that:\n",
         "\n",
         "\\begin{equation}\n",
-        "\\mbox{Norm}_{x}\\Bigl[\\mu, \\sigma^2\\Bigr]=\\mbox{Norm}_{x}\\Bigl[0, 1\\Bigr] \\times \\sigma + \\mu\n",
+        "\\text{Norm}_{x}\\Bigl[\\mu, \\sigma^2\\Bigr]=\\text{Norm}_{x}\\Bigl[0, 1\\Bigr] \\times \\sigma + \\mu\n",
         "\\end{equation}\n",
         "\n",
         "and so:\n",
         "\n",
         "\\begin{equation}\n",
-        "\\mbox{Norm}_{x}\\Bigl[\\phi^3,(\\exp[\\phi])^2\\Bigr]  = \\mbox{Norm}_{x}\\Bigl[0, 1\\Bigr] \\times \\exp[\\phi]+ \\phi^3\n",
+        "\\text{Norm}_{x}\\Bigl[\\phi^3,(\\exp[\\phi])^2\\Bigr]  = \\text{Norm}_{x}\\Bigl[0, 1\\Bigr] \\times \\exp[\\phi]+ \\phi^3\n",
         "\\end{equation}\n",
         "\n",
-        "So, if we draw a sample $\\epsilon^*$ from $\\mbox{Norm}_{\\epsilon}[0, 1]$, then we can compute a sample $x^*$ as:\n",
+        "So, if we draw a sample $\\epsilon^*$ from $\\text{Norm}_{\\epsilon}[0, 1]$, then we can compute a sample $x^*$ as:\n",
         "\n",
-        "\\begin{eqnarray*}\n",
+        "\\begin{align}\n",
         "x^* &=& \\epsilon^* \\times \\sigma + \\mu \\\\\n",
         "&=& \\epsilon^* \\times \\exp[\\phi]+ \\phi^3\n",
-        "\\end{eqnarray*}"
-      ],
-      "metadata": {
-        "id": "zTCykVeWqj_O"
-      }
+        "\\end{align}"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "w13HVpi9q8nF"
+      },
+      "outputs": [],
       "source": [
         "def compute_df_dx_star(x_star):\n",
         "  # TODO Compute this derivative (function defined at the top)\n",
@@ -222,15 +211,15 @@
         "\n",
         "\n",
         "  return df_dphi"
-      ],
-      "metadata": {
-        "id": "w13HVpi9q8nF"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ntQT4An79kAl"
+      },
+      "outputs": [],
       "source": [
         "# Set the seed so the random numbers are all the same\n",
         "np.random.seed(0)\n",
@@ -241,15 +230,15 @@
         "\n",
         "deriv = compute_derivative_of_expectation(phi1, n_samples)\n",
         "print(\"Your value: \", deriv, \", True value:  5.726338035051403\")"
-      ],
-      "metadata": {
-        "id": "ntQT4An79kAl"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "t0Jqd_IN_lMU"
+      },
+      "outputs": [],
       "source": [
         "phi_vals = np.arange(-1.5,1.5, 0.05)\n",
         "deriv_vals = np.zeros_like(phi_vals)\n",
@@ -259,40 +248,38 @@
         "\n",
         "fig,ax = plt.subplots()\n",
         "ax.plot(phi_vals, deriv_vals,'r-')\n",
-        "ax.set_xlabel('Parameter $\\phi$')\n",
-        "ax.set_ylabel('$\\partial/\\partial\\phi\\mathbb{E}_{Pr(x|\\phi)}[f[x]]$')\n",
+        "ax.set_xlabel(r'Parameter $\\phi$')\n",
+        "ax.set_ylabel(r'$\\partial/\\partial\\phi\\mathbb{E}_{Pr(x|\\phi)}[f[x]]$')\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "t0Jqd_IN_lMU"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "This should look plausibly like the derivative of the function we plotted above!"
-      ],
       "metadata": {
         "id": "ASu4yKSwAEYI"
-      }
+      },
+      "source": [
+        "This should look plausibly like the derivative of the function we plotted above!"
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "xoFR1wifc8-b"
+      },
       "source": [
-        "The reparameterization trick computes the derivative of an expectation of a function $\\mbox{f}[x]$:\n",
+        "The reparameterization trick computes the derivative of an expectation of a function $\\text{f}[x]$:\n",
         "\n",
         "\\begin{equation}\n",
-        "\\frac{\\partial}{\\partial \\boldsymbol\\phi} \\mathbb{E}_{Pr(x|\\boldsymbol\\phi)}\\bigl[\\mbox{f}[x]\\bigr],\n",
+        "\\frac{\\partial}{\\partial \\boldsymbol\\phi} \\mathbb{E}_{Pr(x|\\boldsymbol\\phi)}\\bigl[\\text{f}[x]\\bigr],\n",
         "\\end{equation}\n",
         "\n",
         "with respect to the parameters $\\boldsymbol\\phi$ of the distribution $Pr(x|\\boldsymbol\\phi)$ that the expectation is over. This derivative can also be computed as:\n",
         "\n",
-        "\\begin{eqnarray}\n",
-        "\\frac{\\partial}{\\partial \\boldsymbol\\phi} \\mathbb{E}_{Pr(x|\\boldsymbol\\phi)}\\bigl[\\mbox{f}[x]\\bigr] &=& \\mathbb{E}_{Pr(x|\\boldsymbol\\phi)}\\left[\\mbox{f}[x]\\frac{\\partial}{\\partial \\boldsymbol\\phi} \\log\\bigl[ Pr(x|\\boldsymbol\\phi)\\bigr]\\right]\\nonumber \\\\\n",
-        "&\\approx & \\frac{1}{I}\\sum_{i=1}^{I}\\mbox{f}[x_i]\\frac{\\partial}{\\partial \\boldsymbol\\phi} \\log\\bigl[ Pr(x_i|\\boldsymbol\\phi)\\bigr].\n",
-        "\\end{eqnarray}\n",
+        "\\begin{align}\n",
+        "\\frac{\\partial}{\\partial \\boldsymbol\\phi} \\mathbb{E}_{Pr(x|\\boldsymbol\\phi)}\\bigl[\\text{f}[x]\\bigr] &=& \\mathbb{E}_{Pr(x|\\boldsymbol\\phi)}\\left[\\text{f}[x]\\frac{\\partial}{\\partial \\boldsymbol\\phi} \\log\\bigl[ Pr(x|\\boldsymbol\\phi)\\bigr]\\right]\\nonumber \\\\\n",
+        "&\\approx & \\frac{1}{I}\\sum_{i=1}^{I}\\text{f}[x_i]\\frac{\\partial}{\\partial \\boldsymbol\\phi} \\log\\bigl[ Pr(x_i|\\boldsymbol\\phi)\\bigr].\n",
+        "\\end{align}\n",
         "\n",
         "This method is known as the REINFORCE algorithm or score function estimator.  Problem 17.5 asks you to prove this relation.  Let's use this method to compute the gradient and compare.\n",
         "\n",
@@ -301,13 +288,15 @@
         "\\begin{equation}\n",
         " Pr(x|\\mu,\\sigma^2) = \\frac{1}{\\sqrt{2\\pi\\sigma^{2}}}\\exp\\left[-\\frac{(x-\\mu)^{2}}{2\\sigma^{2}}\\right].\n",
         "\\end{equation}\n"
-      ],
-      "metadata": {
-        "id": "xoFR1wifc8-b"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "4TUaxiWvASla"
+      },
+      "outputs": [],
       "source": [
         "def d_log_pr_x_given_phi(x,phi):\n",
         "  # TODO -- fill in this function\n",
@@ -333,15 +322,15 @@
         "\n",
         "\n",
         "  return deriv"
-      ],
-      "metadata": {
-        "id": "4TUaxiWvASla"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "0RSN32Rna_C_"
+      },
+      "outputs": [],
       "source": [
         "# Set the seed so the random numbers are all the same\n",
         "np.random.seed(0)\n",
@@ -352,15 +341,15 @@
         "\n",
         "deriv = compute_derivative_of_expectation_score_function(phi1, n_samples)\n",
         "print(\"Your value: \", deriv, \", True value:  5.724609927313369\")"
-      ],
-      "metadata": {
-        "id": "0RSN32Rna_C_"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "EM_i5zoyElHR"
+      },
+      "outputs": [],
       "source": [
         "phi_vals = np.arange(-1.5,1.5, 0.05)\n",
         "deriv_vals = np.zeros_like(phi_vals)\n",
@@ -370,27 +359,27 @@
         "\n",
         "fig,ax = plt.subplots()\n",
         "ax.plot(phi_vals, deriv_vals,'r-')\n",
-        "ax.set_xlabel('Parameter $\\phi$')\n",
-        "ax.set_ylabel('$\\partial/\\partial\\phi\\mathbb{E}_{Pr(x|\\phi)}[f[x]]$')\n",
+        "ax.set_xlabel(r'Parameter $\\phi$')\n",
+        "ax.set_ylabel(r'$\\partial/\\partial\\phi\\mathbb{E}_{Pr(x|\\phi)}[f[x]]$')\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "EM_i5zoyElHR"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "This should look the same as the derivative that we computed with the reparameterization trick.  So, is there any advantage to one way or the other?  Let's compare the variances of the estimates\n"
-      ],
       "metadata": {
         "id": "1TWBiUC7bQSw"
-      }
+      },
+      "source": [
+        "This should look the same as the derivative that we computed with the reparameterization trick.  So, is there any advantage to one way or the other?  Let's compare the variances of the estimates\n"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "vV_Jx5bCbQGs"
+      },
+      "outputs": [],
       "source": [
         "n_estimate = 100\n",
         "n_sample = 1000\n",
@@ -403,21 +392,31 @@
         "\n",
         "print(\"Variance of reparameterization estimator\", np.var(reparam_estimates))\n",
         "print(\"Variance of score function estimator\", np.var(score_function_estimates))"
-      ],
-      "metadata": {
-        "id": "vV_Jx5bCbQGs"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "The variance of the reparameterization estimator should be quite a bit lower than the score function estimator which is why it is preferred in this situation."
-      ],
       "metadata": {
         "id": "d-0tntSYdKPR"
-      }
+      },
+      "source": [
+        "The variance of the reparameterization estimator should be quite a bit lower than the score function estimator which is why it is preferred in this situation."
+      ]
     }
-  ]
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
 }

Notebooks/Chap17/17_3_Importance_Sampling.ipynb

@@ -1,20 +1,4 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "authorship_tag": "ABX9TyMvae+1cigwg2Htl6vt1Who",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
       "cell_type": "markdown",
@@ -28,33 +12,36 @@
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "t9vk9Elugvmi"
+      },
       "source": [
         "# **Notebook 17.3: Importance sampling**\n",
         "\n",
         "This notebook investigates importance sampling as described in section 17.8.1 of the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
-      ],
-      "metadata": {
-        "id": "t9vk9Elugvmi"
-      }
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "import numpy as np\n",
-        "import matplotlib.pyplot as plt"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "OLComQyvCIJ7"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "import numpy as np\n",
+        "import matplotlib.pyplot as plt"
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "f7a6xqKjkmvT"
+      },
       "source": [
         "Let's approximate the expectation\n",
         "\n",
@@ -65,21 +52,23 @@
         "where\n",
         "\n",
         "\\begin{equation}\n",
-        "Pr(y)=\\mbox{Norm}_y[0,1]\n",
+        "Pr(y)=\\text{Norm}_y[0,1]\n",
         "\\end{equation}\n",
         "\n",
         "by drawing $I$ samples $y_i$ and using the formula:\n",
         "\n",
         "\\begin{equation}\n",
-        "\\mathbb{E}_{y}\\Bigl[\\exp\\bigl[- (y-1)^4\\bigr]\\Bigr] \\approx \\frac{1}{I} \\sum_{i=1}^I \\exp\\bigl[-(y-1)^4 \\bigr]\n",
+        "\\mathbb{E}_{y}\\Bigl[\\exp\\bigl[- (y-1)^4\\bigr]\\Bigr] \\approx \\frac{1}{I} \\sum_{i=1}^I \\exp\\bigl[-(y_i-1)^4 \\bigr]\n",
         "\\end{equation}"
-      ],
-      "metadata": {
-        "id": "f7a6xqKjkmvT"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "VjkzRr8o2ksg"
+      },
+      "outputs": [],
       "source": [
         "def f(y):\n",
         "  return np.exp(-(y-1) *(y-1) *(y-1) * (y-1))\n",
@@ -95,15 +84,15 @@
         "ax.set_xlabel(\"$y$\")\n",
         "ax.legend()\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "VjkzRr8o2ksg"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "LGAKHjUJnWmy"
+      },
+      "outputs": [],
       "source": [
         "def compute_expectation(n_samples):\n",
         "  # TODO -- compute this expectation\n",
@@ -114,15 +103,15 @@
         "\n",
         "\n",
         "  return expectation"
-      ],
-      "metadata": {
-        "id": "LGAKHjUJnWmy"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "nmvixMqgodIP"
+      },
+      "outputs": [],
       "source": [
         "# Set the seed so the random numbers are all the same\n",
         "np.random.seed(0)\n",
@@ -131,26 +120,26 @@
         "n_samples = 100000000\n",
         "expected_f= compute_expectation(n_samples)\n",
         "print(\"Your value: \", expected_f, \", True value:  0.43160702267383166\")"
-      ],
-      "metadata": {
-        "id": "nmvixMqgodIP"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "Jr4UPcqmnXCS"
+      },
       "source": [
         "Let's investigate how the variance of this approximation decreases as we increase the number of samples $N$.\n",
         "\n",
         "\n"
-      ],
-      "metadata": {
-        "id": "Jr4UPcqmnXCS"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "yrDp1ILUo08j"
+      },
+      "outputs": [],
       "source": [
         "def compute_mean_variance(n_sample):\n",
         "  n_estimate = 10000\n",
@@ -158,32 +147,32 @@
         "  for i in range(n_estimate):\n",
         "    estimates[i] = compute_expectation(n_sample.astype(int))\n",
         "  return np.mean(estimates), np.var(estimates)"
-      ],
-      "metadata": {
-        "id": "yrDp1ILUo08j"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "BcUVsodtqdey"
+      },
+      "outputs": [],
       "source": [
         "# Compute the mean and variance for 1,2,... 20 samples\n",
         "n_sample_all = np.array([1.,2,3,4,5,6,7,8,9,10,15,20,25,30,45,50,60,70,80,90,100,150,200,250,300,350,400,450,500])\n",
         "mean_all = np.zeros_like(n_sample_all)\n",
         "variance_all = np.zeros_like(n_sample_all)\n",
         "for i in range(len(n_sample_all)):\n",
-        "  print(\"Computing mean and variance for expectation with %d samples\"%(n_sample_all[i]))\n",
-        "  mean_all[i],variance_all[i] = compute_mean_variance(n_sample_all[i])"
-      ],
-      "metadata": {
-        "id": "BcUVsodtqdey"
-      },
-      "execution_count": null,
-      "outputs": []
+        "  mean_all[i],variance_all[i] = compute_mean_variance(n_sample_all[i])\n",
+        "  print(\"No samples: \", n_sample_all[i], \", Mean: \", mean_all[i], \", Variance: \", variance_all[i])"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "feXmyk0krpUi"
+      },
+      "outputs": [],
       "source": [
         "fig,ax = plt.subplots()\n",
         "ax.semilogx(n_sample_all, mean_all,'r-',label='mean estimate')\n",
@@ -193,38 +182,38 @@
         "ax.plot([0,500],[0.43160702267383166,0.43160702267383166],'k--',label='true value')\n",
         "ax.legend()\n",
         "plt.show()\n"
-      ],
-      "metadata": {
-        "id": "feXmyk0krpUi"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "As you might expect, the more samples that we use to compute the approximate estimate, the lower the variance of the estimate."
-      ],
       "metadata": {
         "id": "XTUpxFlSuOl7"
-      }
+      },
+      "source": [
+        "As you might expect, the more samples that we use to compute the approximate estimate, the lower the variance of the estimate."
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "6hxsl3Pxo1TT"
+      },
       "source": [
         " Now consider the function\n",
         " \\begin{equation}\n",
         " \\mbox{f}[y]= 20.446\\exp\\left[-(y-3)^4\\right],\n",
         " \\end{equation}\n",
         "\n",
-        "which decreases rapidly as we move away from the position $y=4$."
-      ],
-      "metadata": {
-        "id": "6hxsl3Pxo1TT"
-      }
+        "which decreases rapidly as we move away from the position $y=3$."
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "znydVPW7sL4P"
+      },
+      "outputs": [],
       "source": [
         "def f2(y):\n",
         "  return 20.446*np.exp(- (y-3) *(y-3) *(y-3) * (y-3))\n",
@@ -236,46 +225,46 @@
         "ax.set_xlabel(\"$y$\")\n",
         "ax.legend()\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "znydVPW7sL4P"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "G9Xxo0OJsIqD"
+      },
       "source": [
         "Let's again, compute the expectation:\n",
         "\n",
-        "\\begin{eqnarray}\n",
-        "\\mathbb{E}_{y}\\left[\\mbox{f}[y]\\right] &=& \\int \\mbox{f}[y] Pr(y) dy\\\\\n",
-        "&\\approx& \\frac{1}{I} \\mbox{f}[y]\n",
-        "\\end{eqnarray}\n",
+        "\\begin{align}\n",
+        "\\mathbb{E}_{y}\\left[\\text{f}[y]\\right] &=& \\int \\text{f}[y] Pr(y) dy\\\\\n",
+        "&\\approx& \\frac{1}{I} \\text{f}[y]\n",
+        "\\end{align}\n",
         "\n",
-        "where $Pr(y)=\\mbox{Norm}_y[0,1]$ by approximating with samples $y_{i}$.\n"
-      ],
-      "metadata": {
-        "id": "G9Xxo0OJsIqD"
-      }
+        "where $Pr(y)=\\text{Norm}_y[0,1]$ by approximating with samples $y_{i}$.\n"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "l8ZtmkA2vH4y"
+      },
+      "outputs": [],
       "source": [
         "def compute_expectation2(n_samples):\n",
         "  y = np.random.normal(size=(n_samples,1))\n",
         "  expectation = np.mean(f2(y))\n",
         "\n",
         "  return expectation"
-      ],
-      "metadata": {
-        "id": "l8ZtmkA2vH4y"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "dfUQyJ-svZ6F"
+      },
+      "outputs": [],
       "source": [
         "# Set the seed so the random numbers are all the same\n",
         "np.random.seed(0)\n",
@@ -284,26 +273,26 @@
         "n_samples = 100000000\n",
         "expected_f2= compute_expectation2(n_samples)\n",
         "print(\"Expected value: \", expected_f2)"
-      ],
-      "metadata": {
-        "id": "dfUQyJ-svZ6F"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "2sVDqP0BvxqM"
+      },
       "source": [
         "I deliberately chose this function, because it's expectation is roughly the same as for the previous function.\n",
         "\n",
         "Again, let's look at the mean and the  variance of the estimates"
-      ],
-      "metadata": {
-        "id": "2sVDqP0BvxqM"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "mHnILRkOv0Ir"
+      },
+      "outputs": [],
       "source": [
         "def compute_mean_variance2(n_sample):\n",
         "  n_estimate = 10000\n",
@@ -316,17 +305,17 @@
         "mean_all2 = np.zeros_like(n_sample_all)\n",
         "variance_all2 = np.zeros_like(n_sample_all)\n",
         "for i in range(len(n_sample_all)):\n",
-        "  print(\"Computing variance for expectation with %d samples\"%(n_sample_all[i]))\n",
-        "  mean_all2[i], variance_all2[i] = compute_mean_variance2(n_sample_all[i])"
-      ],
-      "metadata": {
-        "id": "mHnILRkOv0Ir"
-      },
-      "execution_count": null,
-      "outputs": []
+        "  mean_all2[i], variance_all2[i] = compute_mean_variance2(n_sample_all[i])\n",
+        "  print(\"No samples: \", n_sample_all[i], \", Mean: \", mean_all2[i], \", Variance: \", variance_all2[i])"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "FkCX-hxxAnsw"
+      },
+      "outputs": [],
       "source": [
         "fig,ax1 = plt.subplots()\n",
         "ax1.semilogx(n_sample_all, mean_all,'r-',label='mean estimate')\n",
@@ -348,39 +337,39 @@
         "ax2.set_title(\"Second function\")\n",
         "ax2.legend()\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "FkCX-hxxAnsw"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "EtBP6NeLwZqz"
+      },
       "source": [
         "You can see that the variance of the estimate of the second function is considerably worse than the estimate of the variance of estimate of the first function\n",
         "\n",
         "TODO:  Think about why this is."
-      ],
-      "metadata": {
-        "id": "EtBP6NeLwZqz"
-      }
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "_wuF-NoQu1--"
+      },
       "source": [
         " Now let's repeat this experiment with the second function, but this time use importance sampling with auxiliary distribution:\n",
         "\n",
         " \\begin{equation}\n",
-        "   q(y)=\\mbox{Norm}_{y}[3,1]\n",
+        "   q(y)=\\text{Norm}_{y}[3,1]\n",
         " \\end{equation}\n"
-      ],
-      "metadata": {
-        "id": "_wuF-NoQu1--"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "jPm0AVYVIDnn"
+      },
+      "outputs": [],
       "source": [
         "def q_y(y):\n",
         "  return (1/np.sqrt(2*np.pi)) * np.exp(-0.5 * (y-3) * (y-3))\n",
@@ -388,22 +377,22 @@
         "def compute_expectation2b(n_samples):\n",
         "  # TODO -- complete this function\n",
         "  # 1. Draw n_samples from auxiliary distribution\n",
-        "  # 2. Compute f[y] for those samples\n",
+        "  # 2. Compute f2[y] for those samples\n",
         "  # 3. Scale the results by pr_y / q_y\n",
         "  # 4. Compute the mean of these weighted samples\n",
         "  # Replace this line\n",
         "  expectation = 0\n",
         "\n",
         "  return expectation"
-      ],
-      "metadata": {
-        "id": "jPm0AVYVIDnn"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "No2ByVvOM2yQ"
+      },
+      "outputs": [],
       "source": [
         "# Set the seed so the random numbers are all the same\n",
         "np.random.seed(0)\n",
@@ -412,15 +401,15 @@
         "n_samples = 100000000\n",
         "expected_f2= compute_expectation2b(n_samples)\n",
         "print(\"Your value: \", expected_f2,\", True value:  0.43163734204459125 \")"
-      ],
-      "metadata": {
-        "id": "No2ByVvOM2yQ"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "6v8Jc7z4M3Mk"
+      },
+      "outputs": [],
       "source": [
         "def compute_mean_variance2b(n_sample):\n",
         "  n_estimate = 10000\n",
@@ -433,17 +422,17 @@
         "mean_all2b = np.zeros_like(n_sample_all)\n",
         "variance_all2b = np.zeros_like(n_sample_all)\n",
         "for i in range(len(n_sample_all)):\n",
-        "  print(\"Computing variance for expectation with %d samples\"%(n_sample_all[i]))\n",
-        "  mean_all2b[i], variance_all2b[i] = compute_mean_variance2b(n_sample_all[i])"
-      ],
-      "metadata": {
-        "id": "6v8Jc7z4M3Mk"
-      },
-      "execution_count": null,
-      "outputs": []
+        "  mean_all2b[i], variance_all2b[i] = compute_mean_variance2b(n_sample_all[i])\n",
+        "  print(\"No samples: \", n_sample_all[i], \", Mean: \", mean_all2b[i], \", Variance: \", variance_all2b[i])"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "C0beD4sNNM3L"
+      },
+      "outputs": [],
       "source": [
         "fig,ax1 = plt.subplots()\n",
         "ax1.semilogx(n_sample_all, mean_all,'r-',label='mean estimate')\n",
@@ -476,21 +465,31 @@
         "ax2.set_title(\"Second function with importance sampling\")\n",
         "ax2.legend()\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "C0beD4sNNM3L"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "You can see that the importance sampling technique has reduced the amount of variance for any given number of samples."
-      ],
       "metadata": {
         "id": "y8rgge9MNiOc"
-      }
+      },
+      "source": [
+        "You can see that the importance sampling technique has reduced the amount of variance for any given number of samples."
+      ]
     }
-  ]
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
 }

Notebooks/Chap18/18_1_Diffusion_Encoder.ipynb

@@ -1,20 +1,4 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "authorship_tag": "ABX9TyMpC8kgLnXx0XQBtwNAQ4jJ",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
       "cell_type": "markdown",
@@ -28,35 +12,40 @@
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "t9vk9Elugvmi"
+      },
       "source": [
         "# **Notebook 18.1: Diffusion Encoder**\n",
         "\n",
         "This notebook investigates the diffusion encoder as described in section 18.2 of the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
-      ],
-      "metadata": {
-        "id": "t9vk9Elugvmi"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "OLComQyvCIJ7"
+      },
+      "outputs": [],
       "source": [
         "import numpy as np\n",
         "import matplotlib.pyplot as plt\n",
         "from matplotlib.colors import ListedColormap\n",
         "from operator import itemgetter"
-      ],
-      "metadata": {
-        "id": "OLComQyvCIJ7"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "4PM8bf6lO0VE"
+      },
+      "outputs": [],
       "source": [
         "#Create pretty colormap as in book\n",
         "my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
@@ -66,28 +55,28 @@
         "b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
         "my_colormap_vals = np.vstack((r,g,b)).transpose()/255.0\n",
         "my_colormap = ListedColormap(my_colormap_vals)"
-      ],
-      "metadata": {
-        "id": "4PM8bf6lO0VE"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ONGRaQscfIOo"
+      },
+      "outputs": [],
       "source": [
         "# Probability distribution for normal\n",
         "def norm_pdf(x, mu, sigma):\n",
         "    return np.exp(-0.5 * (x-mu) * (x-mu) / (sigma * sigma)) / np.sqrt(2*np.pi*sigma*sigma)"
-      ],
-      "metadata": {
-        "id": "ONGRaQscfIOo"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "gZvG0MKhfY8Y"
+      },
+      "outputs": [],
       "source": [
         "# True distribution is a mixture of four Gaussians\n",
         "class TrueDataDistribution:\n",
@@ -108,15 +97,15 @@
         "    mu_list = list(itemgetter(*hidden)(self.mu))\n",
         "    sigma_list = list(itemgetter(*hidden)(self.sigma))\n",
         "    return mu_list + sigma_list * epsilon"
-      ],
-      "metadata": {
-        "id": "gZvG0MKhfY8Y"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "qXmej3TUuQyp"
+      },
+      "outputs": [],
       "source": [
         "# Define ground truth probability distribution that we will model\n",
         "true_dist = TrueDataDistribution()\n",
@@ -130,24 +119,24 @@
         "ax.set_ylim(0,1.0)\n",
         "ax.set_xlim(-3,3)\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "qXmej3TUuQyp"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Let's first implement the forward process"
-      ],
       "metadata": {
         "id": "XHdtfRP47YLy"
-      }
+      },
+      "source": [
+        "Let's first implement the forward process"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "hkApJ2VJlQuk"
+      },
+      "outputs": [],
       "source": [
         "# Do one step of diffusion (equation 18.1)\n",
         "def diffuse_one_step(z_t_minus_1, beta_t):\n",
@@ -157,24 +146,24 @@
         "  z_t = np.zeros_like(z_t_minus_1)\n",
         "\n",
         "  return z_t"
-      ],
-      "metadata": {
-        "id": "hkApJ2VJlQuk"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Now let's run the diffusion process for a whole bunch of samples"
-      ],
       "metadata": {
         "id": "ECAUfHNi9NVW"
-      }
+      },
+      "source": [
+        "Now let's run the diffusion process for a whole bunch of samples"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "M-TY5w9Q8LYW"
+      },
+      "outputs": [],
       "source": [
         "# Generate some samples\n",
         "n_sample = 10000\n",
@@ -192,24 +181,24 @@
         "\n",
         "for t in range(T):\n",
         "  samples[t+1,:] = diffuse_one_step(samples[t,:], beta)"
-      ],
-      "metadata": {
-        "id": "M-TY5w9Q8LYW"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Let's, plot the evolution of a few paths as in figure 18.2"
-      ],
       "metadata": {
         "id": "jYrAW6tN-gJ4"
-      }
+      },
+      "source": [
+        "Let's, plot the evolution of a few paths as in figure 18.2"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "4XU6CDZC_kFo"
+      },
+      "outputs": [],
       "source": [
         "fig, ax = plt.subplots()\n",
         "t_vals = np.arange(0,101,1)\n",
@@ -223,24 +212,24 @@
         "ax.set_xlabel('value')\n",
         "ax.set_ylabel('z_{t}')\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "4XU6CDZC_kFo"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Notice that the samples have a tendencey to move toward the center.  Now let's look at the histogram of the samples at each stage"
-      ],
       "metadata": {
         "id": "SGTYGGevAktz"
-      }
+      },
+      "source": [
+        "Notice that the samples have a tendency to move toward the center.  Now let's look at the histogram of the samples at each stage"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "bn5E5NzL-evM"
+      },
+      "outputs": [],
       "source": [
         "def draw_hist(z_t,title=''):\n",
         "  fig, ax = plt.subplots()\n",
@@ -248,17 +237,17 @@
         "  plt.hist(z_t , bins=np.arange(-3,3, 0.1), density = True)\n",
         "  ax.set_xlim([-3,3])\n",
         "  ax.set_ylim([0,1.0])\n",
-        "  ax.set_title('title')\n",
+        "  ax.set_title(title)\n",
         "  plt.show()"
-      ],
-      "metadata": {
-        "id": "bn5E5NzL-evM"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "pn_XD-EhBlwk"
+      },
+      "outputs": [],
       "source": [
         "draw_hist(samples[0,:],'Original data')\n",
         "draw_hist(samples[5,:],'Time step 5')\n",
@@ -267,33 +256,33 @@
         "draw_hist(samples[40,:],'Time step 40')\n",
         "draw_hist(samples[80,:],'Time step 80')\n",
         "draw_hist(samples[100,:],'Time step 100')"
-      ],
-      "metadata": {
-        "id": "pn_XD-EhBlwk"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "You can clearly see that as the diffusion process continues, the data becomes more Gaussian."
-      ],
       "metadata": {
         "id": "skuLfGl5Czf4"
-      }
+      },
+      "source": [
+        "You can clearly see that as the diffusion process continues, the data becomes more Gaussian."
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Now let's investigate the diffusion kernel as in figure 18.3 of the book.\n"
-      ],
       "metadata": {
         "id": "s37CBSzzK7wh"
-      }
+      },
+      "source": [
+        "Now let's investigate the diffusion kernel as in figure 18.3 of the book.\n"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "vL62Iym0LEtY"
+      },
+      "outputs": [],
       "source": [
         "def diffusion_kernel(x, t, beta):\n",
         "    # TODO -- write this function\n",
@@ -301,15 +290,15 @@
         "    dk_mean = 0.0 ; dk_std = 1.0\n",
         "\n",
         "    return dk_mean, dk_std"
-      ],
-      "metadata": {
-        "id": "vL62Iym0LEtY"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "KtP1KF8wMh8o"
+      },
+      "outputs": [],
       "source": [
         "def draw_prob_dist(x_plot_vals, prob_dist, title=''):\n",
         "  fig, ax = plt.subplots()\n",
@@ -363,47 +352,47 @@
         "    draw_prob_dist(x_plot_vals, diffusion_kernels[20,:],'$q(z_{20}|x)$')\n",
         "    draw_prob_dist(x_plot_vals, diffusion_kernels[40,:],'$q(z_{40}|x)$')\n",
         "    draw_prob_dist(x_plot_vals, diffusion_kernels[80,:],'$q(z_{80}|x)$')"
-      ],
-      "metadata": {
-        "id": "KtP1KF8wMh8o"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "x = -2\n",
-        "compute_and_plot_diffusion_kernels(x, T, beta, my_colormap)"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "g8TcI5wtRQsx"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "x = -2\n",
+        "compute_and_plot_diffusion_kernels(x, T, beta, my_colormap)"
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "TODO -- Run this for different version of $x$ and check that you understand how the graphs change"
-      ],
       "metadata": {
         "id": "-RuN2lR28-hK"
-      }
+      },
+      "source": [
+        "TODO -- Run this for different version of $x$ and check that you understand how the graphs change"
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "n-x6Whz2J_zy"
+      },
       "source": [
         "Finally, let's estimate the marginal distributions empirically and visualize them as in figure 18.4 of the book. This is only tractable because the data is in one dimension and we know the original distribution.\n",
         "\n",
         "The marginal distribution at time t is the sum of the diffusion kernels for each position x, weighted by the probability of seeing that value of x in the true distribution."
-      ],
-      "metadata": {
-        "id": "n-x6Whz2J_zy"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "YzN5duYpg7C-"
+      },
+      "outputs": [],
       "source": [
         "def diffusion_marginal(x_plot_vals, pr_x_true, t, beta):\n",
         "    # If time is zero then marginal is just original distribution\n",
@@ -414,28 +403,28 @@
         "    marginal_at_time_t = np.zeros_like(pr_x_true);\n",
         "\n",
         "\n",
-        "    # TODO Write ths function\n",
+        "    # TODO Write this function\n",
         "    # 1. For each x (value in x_plot_vals):\n",
-        "      # 2. Compute the mean and variance of the diffusion kernel at time t\n",
-        "      # 3. Compute pdf of this Gaussian at every x_plot_val\n",
-        "      # 4. Weight Gaussian by probability at position x and by 0.01 to componensate for bin size\n",
-        "      # 5. Accumulate weighted Gaussian in marginal at time t.\n",
-        "    # 6. Multiply result by 0.01 to compensate for bin size\n",
+        "    # 2. Compute the mean and variance of the diffusion kernel at time t\n",
+        "    # 3. Compute pdf of this Gaussian at every x_plot_val\n",
+        "    # 4. Weight Gaussian by probability at position x and by 0.01 to componensate for bin size\n",
+        "    # 5. Accumulate weighted Gaussian in marginal at time t.\n",
+        "\n",
         "    # Replace this line:\n",
         "    marginal_at_time_t = marginal_at_time_t\n",
         "\n",
         "\n",
         "\n",
         "    return marginal_at_time_t"
-      ],
-      "metadata": {
-        "id": "YzN5duYpg7C-"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "OgEU9sxjRaeO"
+      },
+      "outputs": [],
       "source": [
         "x_plot_vals = np.arange(-3,3,0.01)\n",
         "marginal_distributions = np.zeros((T+1,len(x_plot_vals)))\n",
@@ -460,12 +449,22 @@
         "draw_prob_dist(x_plot_vals, marginal_distributions[0,:],'$q(z_{0})$')\n",
         "draw_prob_dist(x_plot_vals, marginal_distributions[20,:],'$q(z_{20})$')\n",
         "draw_prob_dist(x_plot_vals, marginal_distributions[60,:],'$q(z_{60})$')"
-      ],
-      "metadata": {
-        "id": "OgEU9sxjRaeO"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     }
-  ]
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
 }

Notebooks/Chap18/18_2_1D_Diffusion_Model.ipynb

@@ -1,48 +1,39 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "authorship_tag": "ABX9TyM4DdZDGoP1xGst+Nn+rwvt",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
+      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
-        "id": "view-in-github",
-        "colab_type": "text"
+        "colab_type": "text",
+        "id": "view-in-github"
       },
       "source": [
         "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/Notebooks/Chap18/18_2_1D_Diffusion_Model.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
       ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
+      "metadata": {
+        "id": "t9vk9Elugvmi"
+      },
       "source": [
         "# **Notebook 18.2: 1D Diffusion Model**\n",
         "\n",
         "This notebook investigates the diffusion encoder as described in section 18.3 and 18.4 of the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
-      ],
-      "metadata": {
-        "id": "t9vk9Elugvmi"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "OLComQyvCIJ7"
+      },
+      "outputs": [],
       "source": [
         "import numpy as np\n",
         "import matplotlib.pyplot as plt\n",
@@ -50,15 +41,15 @@
         "from operator import itemgetter\n",
         "from scipy import stats\n",
         "from IPython.display import display, clear_output"
-      ],
-      "metadata": {
-        "id": "OLComQyvCIJ7"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "4PM8bf6lO0VE"
+      },
+      "outputs": [],
       "source": [
         "#Create pretty colormap as in book\n",
         "my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
@@ -68,28 +59,28 @@
         "b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
         "my_colormap_vals = np.vstack((r,g,b)).transpose()/255.0\n",
         "my_colormap = ListedColormap(my_colormap_vals)"
-      ],
-      "metadata": {
-        "id": "4PM8bf6lO0VE"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ONGRaQscfIOo"
+      },
+      "outputs": [],
       "source": [
         "# Probability distribution for normal\n",
         "def norm_pdf(x, mu, sigma):\n",
         "    return np.exp(-0.5 * (x-mu) * (x-mu) / (sigma * sigma)) / np.sqrt(2*np.pi*sigma*sigma)"
-      ],
-      "metadata": {
-        "id": "ONGRaQscfIOo"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "gZvG0MKhfY8Y"
+      },
+      "outputs": [],
       "source": [
         "# True distribution is a mixture of four Gaussians\n",
         "class TrueDataDistribution:\n",
@@ -110,15 +101,15 @@
         "    mu_list = list(itemgetter(*hidden)(self.mu))\n",
         "    sigma_list = list(itemgetter(*hidden)(self.sigma))\n",
         "    return mu_list + sigma_list * epsilon"
-      ],
-      "metadata": {
-        "id": "gZvG0MKhfY8Y"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "iJu_uBiaeUVv"
+      },
+      "outputs": [],
       "source": [
         "# Define ground truth probability distribution that we will model\n",
         "true_dist = TrueDataDistribution()\n",
@@ -133,25 +124,26 @@
         "ax.set_ylim(0,1.0)\n",
         "ax.set_xlim(-3,3)\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "iJu_uBiaeUVv"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
+      "metadata": {
+        "id": "DRHUG_41i4t_"
+      },
       "source": [
         "To train the model to describe this distribution, we'll need to generate pairs of samples drawn from $Pr(z_t|x)$ (diffusion kernel) and $q(z_{t-1}|z_{t},x)$ (equation 18.15).\n",
         "\n"
-      ],
-      "metadata": {
-        "id": "DRHUG_41i4t_"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "x6B8t72Ukscd"
+      },
+      "outputs": [],
       "source": [
         "# The diffusion kernel returns the parameters of Pr(z_{t}|x)\n",
         "def diffusion_kernel(x, t, beta):\n",
@@ -172,7 +164,7 @@
         "    # Find diffusion kernel for every x_train and draw samples\n",
         "    dk_mean, dk_std = diffusion_kernel(x_train, t, beta)\n",
         "    z_t = np.random.normal(size=x_train.shape) * dk_std + dk_mean\n",
-        "    # Find conditional diffusion distribution for each x_train, z pair and draw samlpes\n",
+        "    # Find conditional diffusion distribution for each x_train, z pair and draw samples\n",
         "    cd_mean, cd_std  = conditional_diffusion_distribution(x_train,z_t,t,beta)\n",
         "    if t == 1:\n",
         "      z_tminus1 = x_train\n",
@@ -180,24 +172,25 @@
         "      z_tminus1 = np.random.normal(size=x_train.shape) * cd_std + cd_mean\n",
         "\n",
         "    return z_t, z_tminus1"
-      ],
-      "metadata": {
-        "id": "x6B8t72Ukscd"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "We also need models $\\mbox{f}_t[z_{t},\\phi_{t}]$ that map from $z_{t}$ to the mean of the distribution at time $z_{t-1}$.  We're just going to use a very hacky non-parametric model (basically a lookup table) that tells you the result based on the (quantized) input."
-      ],
       "metadata": {
         "id": "aSG_4uA8_zZ-"
-      }
+      },
+      "source": [
+        "We also need models $\\text{f}_t[z_{t},\\phi_{t}]$ that map from $z_{t}$ to the mean of the distribution at time $z_{t-1}$.  We're just going to use a very hacky non-parametric model (basically a lookup table) that tells you the result based on the (quantized) input."
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ZHViC0pL_yy5"
+      },
+      "outputs": [],
       "source": [
         "# This code is really ugly!  Don't look too closely at it!\n",
         "# All you need to know is that it is a model that trains from pairs zt, zt_minus1\n",
@@ -223,15 +216,15 @@
         "      bin_index = np.floor((zt+self.max_val)/self.inc)\n",
         "      bin_index = np.clip(bin_index,0, len(self.model)-1).astype('uint32')\n",
         "      return zt + self.model[bin_index]"
-      ],
-      "metadata": {
-        "id": "ZHViC0pL_yy5"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "CzVFybWoBygu"
+      },
+      "outputs": [],
       "source": [
         "# Sample data from distribution (this would usually be our collected training set)\n",
         "n_sample = 100000\n",
@@ -249,24 +242,25 @@
         "    all_models.append(NonParametricModel())\n",
         "    # The model at index t maps data from z_{t+1} to z_{t}\n",
         "    all_models[t].train(zt,zt_minus1)"
-      ],
-      "metadata": {
-        "id": "CzVFybWoBygu"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Now that we've learned the model, let's draw some samples from it.  We start at $z_{100}$ and use the model to predict $z_{99}$, then $z_{98}$ and so on until finally we get to $z_{1}$ and then $x$ (represented as $z_{0}$ here). We'll store all of the intermediate stages as well, so we can plot the trajectories.  See equations 18.16."
-      ],
       "metadata": {
         "id": "ZPc9SEvtl14U"
-      }
+      },
+      "source": [
+        "Now that we've learned the model, let's draw some samples from it.  We start at $z_{100}$ and use the model to predict $z_{99}$, then $z_{98}$ and so on until finally we get to $z_{1}$ and then $x$ (represented as $z_{0}$ here). We'll store all of the intermediate stages as well, so we can plot the trajectories.  See equations 18.16."
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "A-ZMFOvACIOw"
+      },
+      "outputs": [],
       "source": [
         "def sample(model, T, sigma_t, n_samples):\n",
         "    # Create the output array\n",
@@ -295,24 +289,25 @@
         "            samples[t-1,:] = samples[t-1,:]\n",
         "\n",
         "    return samples"
-      ],
-      "metadata": {
-        "id": "A-ZMFOvACIOw"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Now let's run the diffusion process for a whole bunch of samples"
-      ],
       "metadata": {
         "id": "ECAUfHNi9NVW"
-      }
+      },
+      "source": [
+        "Now let's run the diffusion process for a whole bunch of samples"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "M-TY5w9Q8LYW"
+      },
+      "outputs": [],
       "source": [
         "sigma_t=0.12288\n",
         "n_samples = 100000\n",
@@ -329,24 +324,25 @@
         "plt.hist(sampled_data, bins=bins, density =True)\n",
         "ax.set_ylim(0, 0.8)\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "M-TY5w9Q8LYW"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Let's, plot the evolution of a few of the paths as in figure 18.7 (paths are from bottom to top now)."
-      ],
       "metadata": {
         "id": "jYrAW6tN-gJ4"
-      }
+      },
+      "source": [
+        "Let's, plot the evolution of a few of the paths as in figure 18.7 (paths are from bottom to top now)."
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "4XU6CDZC_kFo"
+      },
+      "outputs": [],
       "source": [
         "fig, ax = plt.subplots()\n",
         "t_vals = np.arange(0,101,1)\n",
@@ -360,21 +356,33 @@
         "ax.set_xlabel('value')\n",
         "ax.set_ylabel('z_{t}')\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "4XU6CDZC_kFo"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Notice that the samples have a tendency to move from positions that are near the center at time 100 to positions that are high in the true probability distribution at time 0"
-      ],
       "metadata": {
         "id": "SGTYGGevAktz"
-      }
+      },
+      "source": [
+        "Notice that the samples have a tendency to move from positions that are near the center at time 100 to positions that are high in the true probability distribution at time 0"
+      ]
     }
-  ]
-}
+  ],
+  "metadata": {
+    "colab": {
+      "authorship_tag": "ABX9TyM4DdZDGoP1xGst+Nn+rwvt",
+      "include_colab_link": true,
+      "provenance": []
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}

Notebooks/Chap18/18_3_Reparameterized_Model.ipynb

@@ -1,48 +1,39 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "authorship_tag": "ABX9TyNd+D0/IVWXtU2GKsofyk2d",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
+      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
-        "id": "view-in-github",
-        "colab_type": "text"
+        "colab_type": "text",
+        "id": "view-in-github"
       },
       "source": [
         "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/Notebooks/Chap18/18_3_Reparameterized_Model.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
       ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
+      "metadata": {
+        "id": "t9vk9Elugvmi"
+      },
       "source": [
         "# **Notebook 18.3: 1D Reparameterized model**\n",
         "\n",
         "This notebook investigates the reparameterized model as described in section 18.5 of the book and implements algorithms 18.1 and 18.2.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
-      ],
-      "metadata": {
-        "id": "t9vk9Elugvmi"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "OLComQyvCIJ7"
+      },
+      "outputs": [],
       "source": [
         "import numpy as np\n",
         "import matplotlib.pyplot as plt\n",
@@ -50,15 +41,15 @@
         "from operator import itemgetter\n",
         "from scipy import stats\n",
         "from IPython.display import display, clear_output"
-      ],
-      "metadata": {
-        "id": "OLComQyvCIJ7"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "4PM8bf6lO0VE"
+      },
+      "outputs": [],
       "source": [
         "#Create pretty colormap as in book\n",
         "my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
@@ -68,28 +59,28 @@
         "b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
         "my_colormap_vals = np.vstack((r,g,b)).transpose()/255.0\n",
         "my_colormap = ListedColormap(my_colormap_vals)"
-      ],
-      "metadata": {
-        "id": "4PM8bf6lO0VE"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ONGRaQscfIOo"
+      },
+      "outputs": [],
       "source": [
         "# Probability distribution for normal\n",
         "def norm_pdf(x, mu, sigma):\n",
         "    return np.exp(-0.5 * (x-mu) * (x-mu) / (sigma * sigma)) / np.sqrt(2*np.pi*sigma*sigma)"
-      ],
-      "metadata": {
-        "id": "ONGRaQscfIOo"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "gZvG0MKhfY8Y"
+      },
+      "outputs": [],
       "source": [
         "# True distribution is a mixture of four Gaussians\n",
         "class TrueDataDistribution:\n",
@@ -110,15 +101,15 @@
         "    mu_list = list(itemgetter(*hidden)(self.mu))\n",
         "    sigma_list = list(itemgetter(*hidden)(self.sigma))\n",
         "    return mu_list + sigma_list * epsilon"
-      ],
-      "metadata": {
-        "id": "gZvG0MKhfY8Y"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "iJu_uBiaeUVv"
+      },
+      "outputs": [],
       "source": [
         "# Define ground truth probability distribution that we will model\n",
         "true_dist = TrueDataDistribution()\n",
@@ -133,25 +124,26 @@
         "ax.set_ylim(0,1.0)\n",
         "ax.set_xlim(-3,3)\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "iJu_uBiaeUVv"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
+      "metadata": {
+        "id": "DRHUG_41i4t_"
+      },
       "source": [
         "To train the model to describe this distribution, we'll need to generate pairs of samples drawn from $Pr(z_t|x)$ (diffusion kernel) and $q(z_{t-1}|z_{t},x)$ (equation 18.15).\n",
         "\n"
-      ],
-      "metadata": {
-        "id": "DRHUG_41i4t_"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "x6B8t72Ukscd"
+      },
+      "outputs": [],
       "source": [
         "# Return z_t (the argument of g_{t}[] in the loss function in algorithm 18.1) and epsilon\n",
         "def get_data_pairs(x_train,t,beta):\n",
@@ -161,24 +153,25 @@
         "    z_t = np.ones_like(x_train)\n",
         "\n",
         "    return z_t, epsilon"
-      ],
-      "metadata": {
-        "id": "x6B8t72Ukscd"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "We also need models $\\mbox{g}_t[z_{t},\\phi_{t}]$ that map from $z_{t}$ to the noise $\\epsilon$ that was added.  We're just going to use a very hacky non-parametric model (basically a lookup table) that tells you the result based on the (quantized) input."
-      ],
       "metadata": {
         "id": "aSG_4uA8_zZ-"
-      }
+      },
+      "source": [
+        "We also need models $\\text{g}_t[z_{t},\\phi_{t}]$ that map from $z_{t}$ to the noise $\\epsilon$ that was added.  We're just going to use a very hacky non-parametric model (basically a lookup table) that tells you the result based on the (quantized) input."
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ZHViC0pL_yy5"
+      },
+      "outputs": [],
       "source": [
         "# This code is really ugly!  Don't look too closely at it!\n",
         "# All you need to know is that it is a model that trains from pairs zt, zt_minus1\n",
@@ -204,15 +197,15 @@
         "      bin_index = np.floor((zt+self.max_val)/self.inc)\n",
         "      bin_index = np.clip(bin_index,0, len(self.model)-1).astype('uint32')\n",
         "      return self.model[bin_index]"
-      ],
-      "metadata": {
-        "id": "ZHViC0pL_yy5"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "CzVFybWoBygu"
+      },
+      "outputs": [],
       "source": [
         "# Sample data from distribution (this would usually be our collected training set)\n",
         "n_sample = 100000\n",
@@ -230,24 +223,25 @@
         "    all_models.append(NonParametricModel())\n",
         "    # The model at index t maps data from z_{t+1} to epsilon\n",
         "    all_models[t].train(zt,epsilon)"
-      ],
-      "metadata": {
-        "id": "CzVFybWoBygu"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Now that we've learned the model, let's draw some samples from it.  We start at $z_{100}$ and use the model to predict $z_{99}$, then $z_{98}$ and so on until finally we get to $z_{1}$ and then $x$ (represented as $z_{0}$ here). We'll store all of the intermediate stages as well, so we can plot the trajectories.  See algorithm 18.2"
-      ],
       "metadata": {
         "id": "ZPc9SEvtl14U"
-      }
+      },
+      "source": [
+        "Now that we've learned the model, let's draw some samples from it.  We start at $z_{100}$ and use the model to predict $z_{99}$, then $z_{98}$ and so on until finally we get to $z_{1}$ and then $x$ (represented as $z_{0}$ here). We'll store all of the intermediate stages as well, so we can plot the trajectories.  See algorithm 18.2"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "A-ZMFOvACIOw"
+      },
+      "outputs": [],
       "source": [
         "def sample(model, T, sigma_t, n_samples):\n",
         "    # Create the output array\n",
@@ -277,24 +271,25 @@
         "            samples[t-1,:] = samples[t-1,:]\n",
         "\n",
         "    return samples"
-      ],
-      "metadata": {
-        "id": "A-ZMFOvACIOw"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Now let's run the diffusion process for a whole bunch of samples"
-      ],
       "metadata": {
         "id": "ECAUfHNi9NVW"
-      }
+      },
+      "source": [
+        "Now let's run the diffusion process for a whole bunch of samples"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "M-TY5w9Q8LYW"
+      },
+      "outputs": [],
       "source": [
         "sigma_t=0.12288\n",
         "n_samples = 100000\n",
@@ -311,24 +306,25 @@
         "plt.hist(sampled_data, bins=bins, density =True)\n",
         "ax.set_ylim(0, 0.8)\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "M-TY5w9Q8LYW"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Let's, plot the evolution of a few of the paths as in figure 18.7 (paths are from bottom to top now)."
-      ],
       "metadata": {
         "id": "jYrAW6tN-gJ4"
-      }
+      },
+      "source": [
+        "Let's, plot the evolution of a few of the paths as in figure 18.7 (paths are from bottom to top now)."
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "4XU6CDZC_kFo"
+      },
+      "outputs": [],
       "source": [
         "fig, ax = plt.subplots()\n",
         "t_vals = np.arange(0,101,1)\n",
@@ -342,21 +338,33 @@
         "ax.set_xlabel('value')\n",
         "ax.set_ylabel('z_{t}')\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "4XU6CDZC_kFo"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Notice that the samples have a tendency to move from positions that are near the center at time 100 to positions that are high in the true probability distribution at time 0"
-      ],
       "metadata": {
         "id": "SGTYGGevAktz"
-      }
+      },
+      "source": [
+        "Notice that the samples have a tendency to move from positions that are near the center at time 100 to positions that are high in the true probability distribution at time 0"
+      ]
     }
-  ]
-}
+  ],
+  "metadata": {
+    "colab": {
+      "authorship_tag": "ABX9TyNd+D0/IVWXtU2GKsofyk2d",
+      "include_colab_link": true,
+      "provenance": []
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}

Notebooks/Chap18/18_4_Families_of_Diffusion_Models.ipynb

@@ -1,48 +1,39 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "authorship_tag": "ABX9TyNFSvISBXo/Z1l+onknF2Gw",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
+      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
-        "id": "view-in-github",
-        "colab_type": "text"
+        "colab_type": "text",
+        "id": "view-in-github"
       },
       "source": [
         "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/Notebooks/Chap18/18_4_Families_of_Diffusion_Models.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
       ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
+      "metadata": {
+        "id": "t9vk9Elugvmi"
+      },
       "source": [
         "# **Notebook 18.4: Families of diffusion models**\n",
         "\n",
         "This notebook investigates the reparameterized model as described in section 18.5 of the book and computers the results shown in figure 18.10c-f.  These models are based on the paper \"Denoising diffusion implicit models\" which can be found [here](https://arxiv.org/pdf/2010.02502.pdf).\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
-      ],
-      "metadata": {
-        "id": "t9vk9Elugvmi"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "OLComQyvCIJ7"
+      },
+      "outputs": [],
       "source": [
         "import numpy as np\n",
         "import matplotlib.pyplot as plt\n",
@@ -50,15 +41,15 @@
         "from operator import itemgetter\n",
         "from scipy import stats\n",
         "from IPython.display import display, clear_output"
-      ],
-      "metadata": {
-        "id": "OLComQyvCIJ7"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "4PM8bf6lO0VE"
+      },
+      "outputs": [],
       "source": [
         "#Create pretty colormap as in book\n",
         "my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
@@ -68,28 +59,28 @@
         "b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
         "my_colormap_vals = np.vstack((r,g,b)).transpose()/255.0\n",
         "my_colormap = ListedColormap(my_colormap_vals)"
-      ],
-      "metadata": {
-        "id": "4PM8bf6lO0VE"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ONGRaQscfIOo"
+      },
+      "outputs": [],
       "source": [
         "# Probability distribution for normal\n",
         "def norm_pdf(x, mu, sigma):\n",
         "    return np.exp(-0.5 * (x-mu) * (x-mu) / (sigma * sigma)) / np.sqrt(2*np.pi*sigma*sigma)"
-      ],
-      "metadata": {
-        "id": "ONGRaQscfIOo"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "gZvG0MKhfY8Y"
+      },
+      "outputs": [],
       "source": [
         "# True distribution is a mixture of four Gaussians\n",
         "class TrueDataDistribution:\n",
@@ -110,15 +101,15 @@
         "    mu_list = list(itemgetter(*hidden)(self.mu))\n",
         "    sigma_list = list(itemgetter(*hidden)(self.sigma))\n",
         "    return mu_list + sigma_list * epsilon"
-      ],
-      "metadata": {
-        "id": "gZvG0MKhfY8Y"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "iJu_uBiaeUVv"
+      },
+      "outputs": [],
       "source": [
         "# Define ground truth probability distribution that we will model\n",
         "true_dist = TrueDataDistribution()\n",
@@ -133,25 +124,26 @@
         "ax.set_ylim(0,1.0)\n",
         "ax.set_xlim(-3,3)\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "iJu_uBiaeUVv"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
+      "metadata": {
+        "id": "DRHUG_41i4t_"
+      },
       "source": [
         "To train the model to describe this distribution, we'll need to generate pairs of samples drawn from $Pr(z_t|x)$ (diffusion kernel) and $q(z_{t-1}|z_{t},x)$ (equation 18.15).\n",
         "\n"
-      ],
-      "metadata": {
-        "id": "DRHUG_41i4t_"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "x6B8t72Ukscd"
+      },
+      "outputs": [],
       "source": [
         "# Return z_t (the argument of g_{t}[] in the loss function in algorithm 18.1) and epsilon\n",
         "def get_data_pairs(x_train,t,beta):\n",
@@ -161,24 +153,25 @@
         "    z_t = x_train * np.sqrt(alpha_t) + np.sqrt(1-alpha_t) * epsilon\n",
         "\n",
         "    return z_t, epsilon"
-      ],
-      "metadata": {
-        "id": "x6B8t72Ukscd"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "We also need models $\\mbox{g}_t[z_{t},\\phi_{t}]$ that map from $z_{t}$ to the noise $\\epsilon$ that was added.  We're just going to use a very hacky non-parametric model (basically a lookup table) that tells you the result based on the (quantized) input."
-      ],
       "metadata": {
         "id": "aSG_4uA8_zZ-"
-      }
+      },
+      "source": [
+        "We also need models $\\text{g}_t[z_{t},\\phi_{t}]$ that map from $z_{t}$ to the noise $\\epsilon$ that was added.  We're just going to use a very hacky non-parametric model (basically a lookup table) that tells you the result based on the (quantized) input."
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ZHViC0pL_yy5"
+      },
+      "outputs": [],
       "source": [
         "# This code is really ugly!  Don't look too closely at it!\n",
         "# All you need to know is that it is a model that trains from pairs zt, zt_minus1\n",
@@ -204,15 +197,15 @@
         "      bin_index = np.floor((zt+self.max_val)/self.inc)\n",
         "      bin_index = np.clip(bin_index,0, len(self.model)-1).astype('uint32')\n",
         "      return self.model[bin_index]"
-      ],
-      "metadata": {
-        "id": "ZHViC0pL_yy5"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "CzVFybWoBygu"
+      },
+      "outputs": [],
       "source": [
         "# Sample data from distribution (this would usually be our collected training set)\n",
         "n_sample = 100000\n",
@@ -230,15 +223,14 @@
         "    all_models.append(NonParametricModel())\n",
         "    # The model at index t maps data from z_{t+1} to epsilon\n",
         "    all_models[t].train(zt,epsilon)"
-      ],
-      "metadata": {
-        "id": "CzVFybWoBygu"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
+      "metadata": {
+        "id": "ZPc9SEvtl14U"
+      },
       "source": [
         "Now that we've learned the model, let's draw some samples from it.  We start at $z_{100}$ and use the model to predict $z_{99}$, then $z_{98}$ and so on until finally we get to $z_{1}$ and then $x$ (represented as $z_{0}$ here). We'll store all of the intermediate stages as well, so we can plot the trajectories.\n",
         "\n",
@@ -247,17 +239,19 @@
         "One such model is the denoising diffusion implicit model, which has a sampling step:\n",
         "\n",
         "\\begin{equation}\n",
-        "\\mathbf{z}_{t-1} = \\sqrt{\\alpha_{t-1}}\\left(\\frac{\\mathbf{z}_{t}-\\sqrt{1-\\alpha_{t}}\\mbox{g}_t[\\mathbf{z}_{t},\\boldsymbol\\phi]}{\\sqrt{\\alpha_{t}}}\\right) + \\sqrt{1-\\alpha_{t-1}-\\sigma^2}\\mbox{g}_t[\\mathbf{z}_{t},\\boldsymbol\\phi]+\\sigma\\epsilon\n",
+        "\\mathbf{z}_{t-1} = \\sqrt{\\alpha_{t-1}}\\left(\\frac{\\mathbf{z}_{t}-\\sqrt{1-\\alpha_{t}}\\text{g}_t[\\mathbf{z}_{t},\\boldsymbol\\phi]}{\\sqrt{\\alpha_{t}}}\\right) + \\sqrt{1-\\alpha_{t-1}-\\sigma^2}\\text{g}_t[\\mathbf{z}_{t},\\boldsymbol\\phi]+\\sigma\\epsilon\n",
         "\\end{equation}\n",
         "\n",
         "(see equation 12 of the denoising [diffusion implicit models paper ](https://arxiv.org/pdf/2010.02502.pdf).\n"
-      ],
-      "metadata": {
-        "id": "ZPc9SEvtl14U"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "A-ZMFOvACIOw"
+      },
+      "outputs": [],
       "source": [
         "def sample_ddim(model, T, sigma_t, n_samples):\n",
         "    # Create the output array\n",
@@ -283,24 +277,25 @@
         "        if t>0:\n",
         "            samples[t-1,:] = samples[t-1,:]+ np.random.standard_normal(n_samples) * sigma_t\n",
         "    return samples"
-      ],
-      "metadata": {
-        "id": "A-ZMFOvACIOw"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Now let's run the diffusion process for a whole bunch of samples"
-      ],
       "metadata": {
         "id": "ECAUfHNi9NVW"
-      }
+      },
+      "source": [
+        "Now let's run the diffusion process for a whole bunch of samples"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "M-TY5w9Q8LYW"
+      },
+      "outputs": [],
       "source": [
         "# Now we'll set the noise to a MUCH smaller level\n",
         "sigma_t=0.001\n",
@@ -318,24 +313,25 @@
         "plt.hist(sampled_data, bins=bins, density =True)\n",
         "ax.set_ylim(0, 0.8)\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "M-TY5w9Q8LYW"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Let's, plot the evolution of a few of the paths as in figure 18.7 (paths are from bottom to top now)."
-      ],
       "metadata": {
         "id": "jYrAW6tN-gJ4"
-      }
+      },
+      "source": [
+        "Let's, plot the evolution of a few of the paths as in figure 18.7 (paths are from bottom to top now)."
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "4XU6CDZC_kFo"
+      },
+      "outputs": [],
       "source": [
         "fig, ax = plt.subplots()\n",
         "t_vals = np.arange(0,101,1)\n",
@@ -349,35 +345,37 @@
         "ax.set_xlabel('value')\n",
         "ax.set_ylabel('z_{t}')\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "4XU6CDZC_kFo"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "The samples have a tendency to move from positions that are near the center at time 100 to positions that are high in the true probability distribution at time 0"
-      ],
       "metadata": {
         "id": "SGTYGGevAktz"
-      }
+      },
+      "source": [
+        "The samples have a tendency to move from positions that are near the center at time 100 to positions that are high in the true probability distribution at time 0"
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
+      "metadata": {
+        "id": "Z-LZp_fMXxRt"
+      },
       "source": [
         "Let's now sample from the accelerated model, that requires fewer models.  Again, we don't need to learn anything new -- this is just the reverse process that corresponds to a different forward process that is compatible with the same diffusion kernel.\n",
         "\n",
         "There's nothing to do here except read the code.  It uses the same DDIM model as you just implemented in the previous step, but it jumps timesteps five at a time."
-      ],
-      "metadata": {
-        "id": "Z-LZp_fMXxRt"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "3Z0erjGbYj1u"
+      },
+      "outputs": [],
       "source": [
         "def sample_accelerated(model, T, sigma_t, n_steps, n_samples):\n",
         "    # Create the output array\n",
@@ -403,24 +401,25 @@
         "      if t>0:\n",
         "            samples[c_step-1,:] = samples[c_step-1,:]+ np.random.standard_normal(n_samples) * sigma_t\n",
         "    return samples"
-      ],
-      "metadata": {
-        "id": "3Z0erjGbYj1u"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
+      "attachments": {},
       "cell_type": "markdown",
-      "source": [
-        "Now let's draw a bunch of samples from the model"
-      ],
       "metadata": {
         "id": "D3Sm_WYrcuED"
-      }
+      },
+      "source": [
+        "Now let's draw a bunch of samples from the model"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "UB45c7VMcGy-"
+      },
+      "outputs": [],
       "source": [
         "sigma_t=0.11\n",
         "n_samples = 100000\n",
@@ -438,15 +437,15 @@
         "plt.hist(sampled_data, bins=bins, density =True)\n",
         "ax.set_ylim(0, 0.9)\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "UB45c7VMcGy-"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Luv-6w84c_qO"
+      },
+      "outputs": [],
       "source": [
         "fig, ax = plt.subplots()\n",
         "step_increment = 100/ n_steps\n",
@@ -464,21 +463,32 @@
         "ax.set_xlabel('value')\n",
         "ax.set_ylabel('z_{t}')\n",
         "plt.show()"
-      ],
-      "metadata": {
-        "id": "Luv-6w84c_qO"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
-      "source": [],
+      "execution_count": null,
       "metadata": {
         "id": "LSJi72f0kw_e"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": []
     }
-  ]
-}
+  ],
+  "metadata": {
+    "colab": {
+      "authorship_tag": "ABX9TyNFSvISBXo/Z1l+onknF2Gw",
+      "include_colab_link": true,
+      "provenance": []
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}

Notebooks/Chap19/19_1_Markov_Decision_Processes.ipynb

@@ -33,7 +33,7 @@
         "\n",
         "This notebook investigates Markov decision processes as described in section 19.1 of the book.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -598,7 +598,7 @@
       "source": [
         "def markov_decision_process_step_deterministic(state, transition_probabilities_given_action, reward_structure, policy):\n",
         "  # TODO -- complete this function.\n",
-        "  # For each state, theres is a corresponding action.\n",
+        "  # For each state, there's is a corresponding action.\n",
         "  # Draw the next state based on the current state and that action\n",
         "  # and calculate the reward\n",
         "  # Replace this line:\n",
@@ -683,7 +683,7 @@
       "source": [
         "def markov_decision_process_step_stochastic(state, transition_probabilities_given_action, reward_structure, stochastic_policy):\n",
         "  # TODO -- complete this function.\n",
-        "  # For each state, theres is a corresponding distribution over actions\n",
+        "  # For each state, there's is a corresponding distribution over actions\n",
         "  # Draw a sample from that distribution to get the action\n",
         "  # Draw the next state based on the current state and that action\n",
         "  # and calculate the reward\n",
@@ -733,4 +733,4 @@
       "outputs": []
     }
   ]
-}
+}

Notebooks/Chap19/19_4_Temporal_Difference_Methods.ipynb

@@ -1,20 +1,4 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "authorship_tag": "ABX9TyNEAhORON7DFN1dZMhDK/PO",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
       "cell_type": "markdown",
@@ -28,49 +12,59 @@
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "t9vk9Elugvmi"
+      },
       "source": [
         "# **Notebook 19.4: Temporal difference methods**\n",
         "\n",
-        "This notebook investigates temporal differnece methods for  tabular reinforcement learning as described in section 19.3.3 of the book\n",
+        "This notebook investigates temporal difference methods for  tabular reinforcement learning as described in section 19.3.3 of the book\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
-        "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
-      ],
-      "metadata": {
-        "id": "t9vk9Elugvmi"
-      }
+        "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
+        "\n",
+        "Thanks to [Akshil Patel](https://www.akshilpatel.com) and [Jessica Nicholson](https://jessicanicholson1.github.io) for their help in preparing this notebook."
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "import numpy as np\n",
-        "import matplotlib.pyplot as plt\n",
-        "from PIL import Image"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "OLComQyvCIJ7"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "import numpy as np\n",
+        "import matplotlib.pyplot as plt\n",
+        "from PIL import Image\n",
+        "from IPython.display import clear_output\n",
+        "from time import sleep\n",
+        "from copy import deepcopy"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ZsvrUszPLyEG"
+      },
+      "outputs": [],
       "source": [
         "# Get local copies of components of images\n",
         "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Empty.png\n",
         "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Hole.png\n",
         "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Fish.png\n",
         "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Penguin.png"
-      ],
-      "metadata": {
-        "id": "ZsvrUszPLyEG"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Gq1HfJsHN3SB"
+      },
+      "outputs": [],
       "source": [
         "# Ugly class that takes care of drawing pictures like in the book.\n",
         "# You can totally ignore this code!\n",
@@ -253,269 +247,516 @@
         "          self.draw_text(\"%2.2f\"%(state_action_values[3, c_cell]), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'lc','black')\n",
         "\n",
         "    plt.show()"
-      ],
+      ]
+    },
+    {
+      "cell_type": "markdown",
       "metadata": {
-        "id": "Gq1HfJsHN3SB"
+        "id": "JU8gX59o76xM"
       },
-      "execution_count": null,
-      "outputs": []
+      "source": [
+        "# Penguin Ice Environment\n",
+        "\n",
+        "In this implementation we have designed an icy gridworld that a penguin has to traverse to reach the fish found in the bottom right corner.\n",
+        "\n",
+        "## Environment Description\n",
+        "\n",
+        "Consider having to cross an icy surface to reach the yummy fish. In order to achieve this task as quickly as possible, the penguin needs to waddle along as fast as it can whilst simultaneously avoiding falling into the holes.\n",
+        "\n",
+        "In this icy environment the penguin is at one of the discrete cells in the gridworld. The agent starts each episode on a randomly chosen cell. The environment state dynamics are captured by the transition probabilities $Pr(s_{t+1} |s_t, a_t)$ where $s_t$ is the current state, $a_t$ is the action chosen, and $s_{t+1}$ is the next state at decision stage t. At each decision stage, the penguin can move in one of four directions: $a=0$ means try to go upward, $a=1$, right, $a=2$ down and $a=3$ left.\n",
+        "\n",
+        "However, the ice is slippery, so we don't always go the direction we want to: every time the agent chooses an action, with 0.25 probability, the environment changes the action taken to a different action, which is uniformly sampled from the other available actions.\n",
+        "\n",
+        "The rewards are deterministic; the penguin will receive a reward of +3 if it reaches the fish, -2 if it slips into a hole and 0 otherwise.\n",
+        "\n",
+        "Note that as for the states, we've indexed the actions from zero (unlike in the book) so they map to the indices of arrays better"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "eBQ7lTpJQBSe"
+      },
+      "outputs": [],
       "source": [
         "# We're going to work on the problem depicted in figure 19.10a\n",
         "n_rows = 4; n_cols = 4\n",
         "layout = np.zeros(n_rows * n_cols)\n",
         "reward_structure = np.zeros(n_rows * n_cols)\n",
-        "layout[9] = 1 ; reward_structure[9] = -2\n",
-        "layout[10] = 1; reward_structure[10] = -2\n",
-        "layout[14] = 1; reward_structure[14] = -2\n",
-        "layout[15] = 2; reward_structure[15] = 3\n",
+        "layout[9] = 1 ; reward_structure[9] = -2    # Hole\n",
+        "layout[10] = 1; reward_structure[10] = -2   # Hole\n",
+        "layout[14] = 1; reward_structure[14] = -2   # Hole\n",
+        "layout[15] = 2; reward_structure[15] = 3    # Fish\n",
         "initial_state = 0\n",
         "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
         "mdp_drawer.draw(layout, state = initial_state, rewards=reward_structure, draw_state_index = True)"
-      ],
-      "metadata": {
-        "id": "eBQ7lTpJQBSe"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "For clarity, the black numbers are the state number and the red numbers are the reward for being in that state.  Note that the states are indexed from 0 rather than 1 as in the book to make the code neater."
-      ],
       "metadata": {
         "id": "6Vku6v_se2IG"
-      }
+      },
+      "source": [
+        "For clarity, the black numbers are the state number and the red numbers are the reward for being in that state.  Note that the states are indexed from 0 rather than 1 as in the book to make the code neater."
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "Fhc6DzZNOjiC"
+      },
       "source": [
         "Now let's define the state transition function $Pr(s_{t+1}|s_{t},a)$ in full where $a$ is the actions.  Here $a=0$ means try to go upward, $a=1$, right, $a=2$ down and $a=3$ right.  However, the ice is slippery, so we don't always go the direction we want to.\n",
         "\n",
         "Note that as for the states, we've indexed the actions from zero (unlike in the book) so they map to the indices of arrays better"
-      ],
-      "metadata": {
-        "id": "Fhc6DzZNOjiC"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "wROjgnqh76xN"
+      },
+      "outputs": [],
       "source": [
         "transition_probabilities_given_action0 = np.array(\\\n",
-        "[[0.00 , 0.33, 0.00, 0.00,  0.50, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.50 , 0.00, 0.33, 0.00,  0.00, 0.50, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.33, 0.00, 0.50,  0.00, 0.00, 0.50, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.33, 0.00,  0.00, 0.00, 0.00, 0.50,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.50 , 0.00, 0.00, 0.00,  0.00, 0.17, 0.00, 0.00,   0.50, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.34, 0.00, 0.00,  0.25, 0.00, 0.17, 0.00,   0.00, 0.50, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.34, 0.00,  0.00, 0.17, 0.00, 0.25,   0.00, 0.00, 0.50, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.50,  0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.50,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.25, 0.00, 0.00, 0.00,   0.00, 0.17, 0.00, 0.00,   0.75, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.16, 0.00, 0.00,   0.25, 0.00, 0.17, 0.00,   0.00, 0.50, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.16, 0.00,   0.00, 0.17, 0.00, 0.25,   0.00, 0.00, 0.50, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.75 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.25, 0.00, 0.00, 0.00,   0.00, 0.25, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.16, 0.00, 0.00,   0.25, 0.00, 0.25, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.16, 0.00,   0.00, 0.25, 0.00, 0.25 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.25, 0.00 ],\n",
-        "])\n",
+        "[[0.90, 0.05, 0.00, 0.00,  0.85, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.05, 0.85, 0.05, 0.00,  0.00, 0.85, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.05, 0.85, 0.05,  0.00, 0.00, 0.85, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.05, 0.90,  0.00, 0.00, 0.00, 0.85,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.05, 0.00, 0.00, 0.00,  0.05, 0.05, 0.00, 0.00,  0.85, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.05, 0.00, 0.00,  0.05, 0.00, 0.05, 0.00,  0.00, 0.85, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.05, 0.00,  0.00, 0.05, 0.00, 0.05,  0.00, 0.00, 0.85, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.05, 0.05,  0.00, 0.00, 0.00, 0.85,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00,  0.05, 0.05, 0.00, 0.00,  0.85, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.05, 0.00, 0.00,  0.05, 0.00, 0.05, 0.00,  0.00, 0.85, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.05, 0.00,  0.00, 0.05, 0.00, 0.05,  0.00, 0.00, 0.85, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.05, 0.05,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00,  0.10, 0.05, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.05, 0.00, 0.00,  0.05, 0.05, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.05, 0.00,  0.00, 0.05, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.05, 0.00]])\n",
+        "\n",
         "\n",
         "transition_probabilities_given_action1 = np.array(\\\n",
-        "[[0.00 , 0.25, 0.00, 0.00,  0.25, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.75 , 0.00, 0.25, 0.00,  0.00, 0.17, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.50, 0.00, 0.50,  0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.50, 0.00,  0.00, 0.00, 0.00, 0.33,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.25 , 0.00, 0.00, 0.00,  0.00, 0.17, 0.00, 0.00,   0.25, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.25, 0.00, 0.00,  0.50, 0.00, 0.17, 0.00,   0.00, 0.17, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.25, 0.00,  0.00, 0.50, 0.00, 0.33,   0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.50,  0.00, 0.00, 0.50, 0.00,   0.00, 0.00, 0.00, 0.33,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.25, 0.00, 0.00, 0.00,   0.00, 0.17, 0.00, 0.00,   0.25, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.16, 0.00, 0.00,   0.50, 0.00, 0.17, 0.00,   0.00, 0.25, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.16, 0.00,   0.00, 0.50, 0.00, 0.33,   0.00, 0.00, 0.25, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.34,   0.00, 0.00, 0.50, 0.00,   0.00, 0.00, 0.00, 0.50 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.25, 0.00, 0.00, 0.00,   0.00, 0.25, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.16, 0.00, 0.00,   0.75, 0.00, 0.25, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.16, 0.00,   0.00, 0.50, 0.00, 0.50 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.34,   0.00, 0.00, 0.50, 0.00 ],\n",
-        "])\n",
+        "[[0.10, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.85, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.85, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.85, 0.90, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.05, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.85, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.85, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.10, 0.05, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.05, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00]])\n",
+        "\n",
         "\n",
         "transition_probabilities_given_action2 = np.array(\\\n",
-        "[[0.00 , 0.25, 0.00, 0.00,  0.25, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.25 , 0.00, 0.25, 0.00,  0.00, 0.17, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.25, 0.00, 0.25,  0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.25, 0.00,  0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.75 , 0.00, 0.00, 0.00,  0.00, 0.17, 0.00, 0.00,   0.25, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.50, 0.00, 0.00,  0.25, 0.00, 0.17, 0.00,   0.00, 0.17, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.50, 0.00,  0.00, 0.16, 0.00, 0.25,   0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.75,  0.00, 0.00, 0.16, 0.00,   0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.50, 0.00, 0.00, 0.00,   0.00, 0.17, 0.00, 0.00,   0.50, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.50, 0.00, 0.00,   0.25, 0.00, 0.17, 0.00,   0.00, 0.33, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.50, 0.00,   0.00, 0.16, 0.00, 0.25,   0.00, 0.00, 0.33, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.50,   0.00, 0.00, 0.16, 0.00,   0.00, 0.00, 0.00, 0.50 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.50, 0.00, 0.00, 0.00,   0.00, 0.33, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.50, 0.00, 0.00,   0.50, 0.00, 0.33, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.50, 0.00,   0.00, 0.34, 0.00, 0.50 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.50,   0.00, 0.00, 0.34, 0.00 ],\n",
-        "])\n",
+        "[[0.10, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.05, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.05, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.05, 0.10, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.85, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.90, 0.05, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.85, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.85, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00]])\n",
         "\n",
         "transition_probabilities_given_action3 = np.array(\\\n",
-        "[[0.00 , 0.25, 0.00, 0.00,  0.33, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.50 , 0.00, 0.25, 0.00,  0.00, 0.17, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.50, 0.00, 0.75,  0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.50, 0.00,  0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.50 , 0.00, 0.00, 0.00,  0.00, 0.50, 0.00, 0.00,   0.33, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.25, 0.00, 0.00,  0.33, 0.00, 0.50, 0.00,   0.00, 0.17, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.25, 0.00,  0.00, 0.17, 0.00, 0.50,   0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.25,  0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.34, 0.00, 0.00, 0.00,   0.00, 0.50, 0.00, 0.00,   0.50, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.16, 0.00, 0.00,   0.33, 0.00, 0.50, 0.00,   0.00, 0.25, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.16, 0.00,   0.00, 0.17, 0.00, 0.50,   0.00, 0.00, 0.25, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.25 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.34, 0.00, 0.00, 0.00,   0.00, 0.50, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.16, 0.00, 0.00,   0.50, 0.00, 0.50, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.16, 0.00,   0.00, 0.25, 0.00, 0.75 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.25, 0.00 ],\n",
-        "])\n",
+        "[[0.90, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.05, 0.05, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.05, 0.05, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.05, 0.10, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.05, 0.00, 0.00, 0.00, 0.85, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.85, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.90, 0.85, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.85, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00]])\n",
+        "\n",
+        "\n",
         "\n",
         "# Store all of these in a three dimension array\n",
         "# Pr(s_{t+1}=2|s_{t}=1, a_{t}=3] is stored at position [2,1,3]\n",
         "transition_probabilities_given_action = np.concatenate((np.expand_dims(transition_probabilities_given_action0,2),\n",
         "                                                        np.expand_dims(transition_probabilities_given_action1,2),\n",
         "                                                        np.expand_dims(transition_probabilities_given_action2,2),\n",
-        "                                                        np.expand_dims(transition_probabilities_given_action3,2)),axis=2)"
-      ],
+        "                                                        np.expand_dims(transition_probabilities_given_action3,2)),axis=2)\n",
+        "\n",
+        "print('Grid Size:', len(transition_probabilities_given_action[0]))\n",
+        "print()\n",
+        "print('Transition Probabilities for when next state = 2:')\n",
+        "print(transition_probabilities_given_action[2])\n",
+        "print()\n",
+        "print('Transitions Probabilities for when next state = 2 and current state = 1')\n",
+        "print(transition_probabilities_given_action[2][1])\n",
+        "print()\n",
+        "print('Transitions Probabilities for  when next state = 2 and current state = 1 and action = 3 (Left):')\n",
+        "print(transition_probabilities_given_action[2][1][3])"
+      ]
+    },
+    {
+      "cell_type": "markdown",
       "metadata": {
-        "id": "l7rT78BbOgTi"
+        "id": "eblSQ6xZ76xN"
       },
-      "execution_count": null,
-      "outputs": []
+      "source": [
+        "## Implementation Details\n",
+        "\n",
+        "We provide the following methods:\n",
+        "- **`markov_decision_process_step`** - this function simulates $Pr(s_{t+1} | s_{t}, a_{t})$. It randomly selects an action, updates the state based on the transition probabilities associated with the chosen action, and returns the new state, the reward obtained for leaving the current state, and the chosen action. The randomness in action selection and state transitions reflects a random exploration process and the stochastic nature of the MDP, respectively.\n",
+        "\n",
+        "- **`get_policy`** - this function computes a policy that acts greedily with respect to the state-action values. The policy is computed for all states and the action that maximizes the state-action value is chosen for each state. When there are multiple optimal actions, one is chosen at random.\n",
+        "\n",
+        "\n",
+        "You have to implement the following method:\n",
+        "\n",
+        "- **`q_learning_step`** - this function implements a single step of the Q-learning algorithm for reinforcement learning as shown below. The update follows the Q-learning formula and is controlled by parameters such as the learning rate (alpha) and the discount factor $(\\gamma)$. The function returns the updated state-action values matrix.\n",
+        "\n",
+        "$Q(s, a) \\leftarrow (1 - \\alpha) \\cdot Q(s, a) + \\alpha \\cdot \\left(r + \\gamma \\cdot \\max_{a'} Q(s', a')\\right)$"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "cKLn4Iam76xN"
+      },
+      "outputs": [],
       "source": [
-        "def q_learning_step(state_action_values, reward, state, new_state, action, gamma, alpha = 0.1):\n",
+        "def get_policy(state_action_values):\n",
+        "    policy = np.zeros(state_action_values.shape[1]) # One action for each state\n",
+        "    for state in range(state_action_values.shape[1]):\n",
+        "        # Break ties for maximising actions randomly\n",
+        "        policy[state] = np.random.choice(np.flatnonzero(state_action_values[:, state] == max(state_action_values[:, state])))\n",
+        "    return policy"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "akjrncMF-FkU"
+      },
+      "outputs": [],
+      "source": [
+        "def markov_decision_process_step(state, transition_probabilities_given_action, reward_structure, terminal_states, action=None):\n",
+        "  # Pick action\n",
+        "  if action is None:\n",
+        "    action = np.random.randint(4)\n",
+        "  # Update the state\n",
+        "  new_state = np.random.choice(a=range(transition_probabilities_given_action.shape[0]), p = transition_probabilities_given_action[:, state,action])\n",
+        "\n",
+        "  # Return the reward -- here the reward is for arriving at the state\n",
+        "  reward = reward_structure[new_state]\n",
+        "  is_terminal = new_state in terminal_states\n",
+        "\n",
+        "  return new_state, reward, action, is_terminal"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "5pO6-9ACWhiV"
+      },
+      "outputs": [],
+      "source": [
+        "def q_learning_step(state_action_values, reward, state, new_state, action, is_terminal, gamma, alpha = 0.1):\n",
         "  # TODO -- write this function\n",
         "  # Replace this line\n",
         "  state_action_values_after = np.copy(state_action_values)\n",
         "\n",
         "  return state_action_values_after"
-      ],
-      "metadata": {
-        "id": "5pO6-9ACWhiV"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
-      "cell_type": "code",
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "u4OHTTk176xO"
+      },
       "source": [
-        "# This takes a single step from an MDP which just has a completely random policy\n",
-        "def markov_decision_process_step(state, transition_probabilities_given_action, reward_structure):\n",
-        "  # Pick action\n",
-        "  action = np.random.randint(4)\n",
-        "  # Update the state\n",
-        "  new_state = np.random.choice(a=np.arange(0,transition_probabilities_given_action.shape[0]),p = transition_probabilities_given_action[:,state,action])\n",
-        "  # Return the reward -- here the reward is for leaving the state\n",
-        "  reward = reward_structure[state]\n",
-        "\n",
-        "  return new_state, reward, action"
-      ],
-      "metadata": {
-        "id": "akjrncMF-FkU"
-      },
-      "execution_count": null,
-      "outputs": []
+        "Lets run this for a single Q-learning step"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Fu5_VjvbSwfJ"
+      },
+      "outputs": [],
       "source": [
         "# Initialize the state-action values to random numbers\n",
         "np.random.seed(0)\n",
         "n_state = transition_probabilities_given_action.shape[0]\n",
         "n_action = transition_probabilities_given_action.shape[2]\n",
+        "terminal_states=[15]\n",
         "state_action_values = np.random.normal(size=(n_action, n_state))\n",
+        "# Hard code value of termination state of finding fish to 0\n",
+        "state_action_values[:, terminal_states] = 0\n",
         "gamma = 0.9\n",
         "\n",
-        "policy = np.argmax(state_action_values, axis=0).astype(int)\n",
+        "policy = get_policy(state_action_values)\n",
         "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
         "mdp_drawer.draw(layout, policy = policy, state_action_values = state_action_values, rewards = reward_structure)\n",
         "\n",
         "# Now let's simulate a single Q-learning step\n",
         "initial_state = 9\n",
-        "print(\"Initial state = \", initial_state)\n",
-        "new_state, reward, action = markov_decision_process_step(initial_state, transition_probabilities_given_action, reward_structure)\n",
-        "print(\"Action = \", action)\n",
-        "print(\"New state = \", new_state)\n",
-        "print(\"Reward = \", reward)\n",
+        "print(\"Initial state =\",initial_state)\n",
+        "new_state, reward, action, is_terminal = markov_decision_process_step(initial_state, transition_probabilities_given_action, reward_structure, terminal_states)\n",
+        "print(\"Action =\",action)\n",
+        "print(\"New state =\",new_state)\n",
+        "print(\"Reward =\", reward)\n",
         "\n",
-        "state_action_values_after = q_learning_step(state_action_values, reward, initial_state, new_state, action, gamma)\n",
+        "state_action_values_after = q_learning_step(state_action_values, reward, initial_state, new_state, action, is_terminal, gamma)\n",
         "print(\"Your value:\",state_action_values_after[action, initial_state])\n",
-        "print(\"True value:  0.27650262412468796\")\n",
+        "print(\"True value: 0.3024718977397814\")\n",
         "\n",
-        "policy = np.argmax(state_action_values, axis=0).astype(int)\n",
+        "policy = get_policy(state_action_values)\n",
         "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
         "mdp_drawer.draw(layout, policy = policy, state_action_values = state_action_values_after, rewards = reward_structure)\n"
-      ],
-      "metadata": {
-        "id": "Fu5_VjvbSwfJ"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Now let's run this for a while and watch the policy improve"
-      ],
       "metadata": {
         "id": "Ogh0qucmb68J"
-      }
+      },
+      "source": [
+        "Now let's run this for a while (20000) steps and watch the policy improve"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "N6gFYifh76xO"
+      },
+      "outputs": [],
       "source": [
         "# Initialize the state-action values to random numbers\n",
         "np.random.seed(0)\n",
-        "n_state = transition_probabilities_given_action.shape[0]\n",
+        "n_state  = transition_probabilities_given_action.shape[0]\n",
         "n_action = transition_probabilities_given_action.shape[2]\n",
         "state_action_values = np.random.normal(size=(n_action, n_state))\n",
-        "# Hard code termination state of finding fish\n",
-        "state_action_values[:,n_state-1] = 3.0\n",
+        "\n",
+        "# Hard code value of termination state of finding fish to 0\n",
+        "terminal_states = [15]\n",
+        "state_action_values[:, terminal_states] = 0\n",
         "gamma = 0.9\n",
         "\n",
         "# Draw the initial setup\n",
-        "policy = np.argmax(state_action_values, axis=0).astype(int)\n",
+        "print('Initial Policy:')\n",
+        "policy = get_policy(state_action_values)\n",
         "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
         "mdp_drawer.draw(layout, policy = policy, state_action_values = state_action_values, rewards = reward_structure)\n",
         "\n",
-        "\n",
-        "state= np.random.randint(n_state-1)\n",
+        "state = np.random.randint(n_state-1)\n",
         "\n",
         "# Run for a number of iterations\n",
-        "for c_iter in range(10000):\n",
-        "  new_state, reward, action = markov_decision_process_step(state, transition_probabilities_given_action, reward_structure)\n",
-        "  state_action_values_after = q_learning_step(state_action_values, reward, state, new_state, action, gamma)\n",
+        "for c_iter in range(20000):\n",
+        "  new_state, reward, action, is_terminal = markov_decision_process_step(state, transition_probabilities_given_action, reward_structure, terminal_states)\n",
+        "  state_action_values_after = q_learning_step(state_action_values, reward, state, new_state, action, is_terminal, gamma)\n",
+        "\n",
         "  # If in termination state, reset state randomly\n",
-        "  if new_state==15:\n",
-        "    state= np.random.randint(n_state-1)\n",
+        "  if is_terminal:\n",
+        "    state = np.random.randint(n_state-1)\n",
         "  else:\n",
         "    state = new_state\n",
-        "  # Update the policy\n",
-        "  state_action_values = np.copy(state_action_values_after)\n",
-        "  policy = np.argmax(state_action_values, axis=0).astype(int)\n",
         "\n",
+        "  # Update the policy\n",
+        "  state_action_values = deepcopy(state_action_values_after)\n",
+        "  policy = get_policy(state_action_values_after)\n",
+        "\n",
+        "print('Final Optimal Policy:')\n",
         "# Draw the final situation\n",
         "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
         "mdp_drawer.draw(layout, policy = policy, state_action_values = state_action_values, rewards = reward_structure)"
-      ],
+      ]
+    },
+    {
+      "cell_type": "markdown",
       "metadata": {
-        "id": "qQFhwVqPcCFH"
+        "id": "djPTKuDk76xO"
       },
+      "source": [
+        "Finally, lets run this for a **single** episode and visualize the penguin's actions"
+      ]
+    },
+    {
+      "cell_type": "code",
       "execution_count": null,
-      "outputs": []
+      "metadata": {
+        "id": "pWObQf2h76xO"
+      },
+      "outputs": [],
+      "source": [
+        "def get_one_episode(n_state, state_action_values, terminal_states, gamma):\n",
+        "\n",
+        "    state = np.random.randint(n_state-1)\n",
+        "\n",
+        "    # Create lists to store all the states seen and actions taken throughout the single episode\n",
+        "    all_states  = []\n",
+        "    all_actions = []\n",
+        "\n",
+        "    # Initalize episode termination flag\n",
+        "    done  = False\n",
+        "    # Initialize counter for steps in the episode\n",
+        "    steps = 0\n",
+        "\n",
+        "    all_states.append(state)\n",
+        "\n",
+        "    while not done:\n",
+        "      steps += 1\n",
+        "\n",
+        "      new_state, reward, action, is_terminal = markov_decision_process_step(state, transition_probabilities_given_action, reward_structure, terminal_states)\n",
+        "      all_states.append(new_state)\n",
+        "      all_actions.append(action)\n",
+        "\n",
+        "      state_action_values_after = q_learning_step(state_action_values, reward, state, new_state, action, is_terminal, gamma)\n",
+        "\n",
+        "      # If in termination state, reset state randomly\n",
+        "      if is_terminal:\n",
+        "        state = np.random.randint(n_state-1)\n",
+        "        print(f'Episode Terminated at {steps} Steps')\n",
+        "        # Set episode termination flag\n",
+        "        done = True\n",
+        "      else:\n",
+        "        state = new_state\n",
+        "\n",
+        "      # Update the policy\n",
+        "      state_action_values = deepcopy(state_action_values_after)\n",
+        "      policy = get_policy(state_action_values_after)\n",
+        "\n",
+        "    return all_states, all_actions, policy, state_action_values\n",
+        ""
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "P7cbCGT176xO"
+      },
+      "outputs": [],
+      "source": [
+        "def visualize_one_episode(states, actions):\n",
+        "    # Define actions for visualization\n",
+        "    acts = ['up', 'right', 'down', 'left']\n",
+        "\n",
+        "    # Iterate over the states and actions\n",
+        "    for i in range(len(states)):\n",
+        "\n",
+        "        if i == 0:\n",
+        "            print('Starting State:', states[i])\n",
+        "\n",
+        "        elif i == len(states)-1:\n",
+        "            print('Episode Done:', states[i])\n",
+        "\n",
+        "        else:\n",
+        "            print('State', states[i-1])\n",
+        "            a = actions[i]\n",
+        "            print('Action:', acts[a])\n",
+        "            print('Next State:', states[i])\n",
+        "\n",
+        "        # Visualize the current state using the MDP drawer\n",
+        "        mdp_drawer.draw(layout, state=states[i], rewards=reward_structure, draw_state_index=True)\n",
+        "        clear_output(True)\n",
+        "\n",
+        "        # Pause for a short duration to allow observation\n",
+        "        sleep(1.5)\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "cr98F8PT76xP"
+      },
+      "outputs": [],
+      "source": [
+        "# Initialize the state-action values to random numbers\n",
+        "np.random.seed(2)\n",
+        "n_state  = transition_probabilities_given_action.shape[0]\n",
+        "n_action = transition_probabilities_given_action.shape[2]\n",
+        "state_action_values = np.random.normal(size=(n_action, n_state))\n",
+        "\n",
+        "# Hard code value of termination state of finding fish to 0\n",
+        "terminal_states = [15]\n",
+        "state_action_values[:, terminal_states] = 0\n",
+        "gamma = 0.9\n",
+        "\n",
+        "# Draw the initial setup\n",
+        "print('Initial Policy:')\n",
+        "policy = get_policy(state_action_values)\n",
+        "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+        "mdp_drawer.draw(layout, policy = policy, state_action_values = state_action_values, rewards = reward_structure)\n",
+        "\n",
+        "states, actions, policy, state_action_values = get_one_episode(n_state, state_action_values, terminal_states, gamma)\n",
+        "\n",
+        "print()\n",
+        "print('Final Optimal Policy:')\n",
+        "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+        "mdp_drawer.draw(layout, policy = policy, state_action_values = state_action_values, rewards = reward_structure)\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "5zBu1g3776xP"
+      },
+      "outputs": [],
+      "source": [
+        "visualize_one_episode(states, actions)"
+      ]
     }
-  ]
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3 (ipykernel)",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.10.12"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
 }

Notebooks/Chap19/19_5_Control_Variates.ipynb

@@ -34,7 +34,7 @@
         "This notebook investigates the method of control variates as described in figure 19.16\n",
         "\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
       ],
@@ -57,7 +57,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Genearate from our two variables, $a$ and $b$.  We are interested in estimating the mean of $a$, but we can use $b$$ to improve our estimates if it is correlated"
+        "Generate from our two variables, $a$ and $b$.  We are interested in estimating the mean of $a$, but we can use $b$$ to improve our estimates if it is correlated"
       ],
       "metadata": {
         "id": "uwmhcAZBzTRO"

Notebooks/Chap20/20_1_Random_Data.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyPkSYbEjOcEmLt8tU6HxNuR",
+      "authorship_tag": "ABX9TyNgBRvfIlngVobKuLE6leM+",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -33,7 +33,7 @@
         "\n",
         "This notebook investigates training the network with random data, as illustrated in figure 20.1.\n",
         "\n",
-        "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+        "Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
         "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
         "\n"
@@ -45,8 +45,8 @@
     {
       "cell_type": "code",
       "source": [
-        "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
-        "!git clone https://github.com/greydanus/mnist1d"
+        "# Run this if you're in a Colab to install MNIST 1D repository\n",
+        "!pip install git+https://github.com/greydanus/mnist1d"
       ],
       "metadata": {
         "id": "D5yLObtZCi9J"