![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Gradient flow\n",
+ "\n",
+ "This notebook replicates some of the results in the the Borealis AI [blog](https://www.borealisai.com/research-blogs/gradient-flow/) on gradient flow. \n"
+ ],
+ "metadata": {
+ "id": "ucrRRJ4dq8_d"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Import relevant libraries\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from scipy.linalg import expm\n",
+ "from matplotlib import cm\n",
+ "from matplotlib.colors import ListedColormap"
+ ],
+ "metadata": {
+ "id": "_IQFHZEMZE8T"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Create the three data points that are used to train the linear model in the blog. Each input point is a column in $\\mathbf{X}$ and consists of the $x$ position in the plot and the value 1, which is used to allow the model to fit bias terms neatly."
+ ],
+ "metadata": {
+ "id": "NwgUP3MSriiJ"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "cJNZ2VIcYsD8"
+ },
+ "outputs": [],
+ "source": [
+ "X = np.array([[0.2, 0.4, 0.8],[1,1,1]])\n",
+ "y = np.array([[-0.1],[0.15],[0.3]])\n",
+ "D = X.shape[0]\n",
+ "I = X.shape[1]\n",
+ "\n",
+ "print(\"X=\\n\",X)\n",
+ "print(\"y=\\n\",y)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Draw the three data points\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(X[0:1,:],y.T,'ro')\n",
+ "ax.set_xlim([0,1]); ax.set_ylim([-0.5,0.5])\n",
+ "ax.set_xlabel('x'); ax.set_ylabel('y')\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "FpFlD4nUZDRt"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Compute the evolution of the residuals, loss, and parameters as a function of time."
+ ],
+ "metadata": {
+ "id": "H2LBR1DasQej"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Discretized time to evaluate quantities at\n",
+ "t_all = np.arange(0,20,0.01)\n",
+ "nT = t_all.shape[0]\n",
+ "\n",
+ "# Initial parameters, and initial function output at training points\n",
+ "phi_0 = np.array([[-0.05],[-0.4]])\n",
+ "f_0 = X.T @ phi_0\n",
+ "\n",
+ "# Precompute pseudoinverse term (not a very sensible numerical implementation, but it works...)\n",
+ "XXTInvX = np.linalg.inv(X@X.T)@X\n",
+ "\n",
+ "# Create arrays to hold function at data points over time, residual over time, parameters over time\n",
+ "f_all = np.zeros((I,nT))\n",
+ "f_minus_y_all = np.zeros((I,nT))\n",
+ "phi_t_all = np.zeros((D,nT))\n",
+ "\n",
+ "# For each time, compute function, residual, and parameters at each time.\n",
+ "for t in range(len(t_all)):\n",
+ " f = y + expm(-X.T@X * t_all[t]) @ (f_0-y)\n",
+ " f_all[:,t:t+1] = f\n",
+ " f_minus_y_all[:,t:t+1] = f-y\n",
+ " phi_t_all[:,t:t+1] = phi_0 - XXTInvX @ (np.identity(3)-expm(-X.T@X * t_all[t])) @ (f_0-y)"
+ ],
+ "metadata": {
+ "id": "wfF_oTS5Z4Wi"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Plot the results that were calculated in the previous cell"
+ ],
+ "metadata": {
+ "id": "9jSjOOFutJUE"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Plot function at data points\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(t_all,np.squeeze(f_all[0,:]),'r-', label='$f[x_{0},\\phi]$')\n",
+ "ax.plot(t_all,np.squeeze(f_all[1,:]),'g-', label='$f[x_{1},\\phi]$')\n",
+ "ax.plot(t_all,np.squeeze(f_all[2,:]),'b-', label='$f[x_{2},\\phi]$')\n",
+ "ax.set_xlim([0,np.max(t_all)]); ax.set_ylim([-0.5,0.5])\n",
+ "ax.set_xlabel('t'); ax.set_ylabel('f')\n",
+ "plt.legend(loc=\"lower right\")\n",
+ "plt.show()\n",
+ "\n",
+ "# Plot residual\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(t_all,np.squeeze(f_minus_y_all[0,:]),'r-', label='$f[x_{0},\\phi]-y_{0}$')\n",
+ "ax.plot(t_all,np.squeeze(f_minus_y_all[1,:]),'g-', label='$f[x_{1},\\phi]-y_{1}$')\n",
+ "ax.plot(t_all,np.squeeze(f_minus_y_all[2,:]),'b-', label='$f[x_{2},\\phi]-y_{2}$')\n",
+ "ax.set_xlim([0,np.max(t_all)]); ax.set_ylim([-0.5,0.5])\n",
+ "ax.set_xlabel('t'); ax.set_ylabel('f-y')\n",
+ "plt.legend(loc=\"lower right\")\n",
+ "plt.show()\n",
+ "\n",
+ "# Plot loss (sum of residuals)\n",
+ "fig, ax = plt.subplots()\n",
+ "square_error = 0.5 * np.sum(f_minus_y_all * f_minus_y_all, axis=0)\n",
+ "ax.plot(t_all, square_error,'k-')\n",
+ "ax.set_xlim([0,np.max(t_all)]); ax.set_ylim([-0.0,0.25])\n",
+ "ax.set_xlabel('t'); ax.set_ylabel('Loss')\n",
+ "plt.show()\n",
+ "\n",
+ "# Plot parameters\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(t_all, np.squeeze(phi_t_all[0,:]),'c-',label='$\\phi_{0}$')\n",
+ "ax.plot(t_all, np.squeeze(phi_t_all[1,:]),'m-',label='$\\phi_{1}$')\n",
+ "ax.set_xlim([0,np.max(t_all)]); ax.set_ylim([-1,1])\n",
+ "ax.set_xlabel('t'); ax.set_ylabel('$\\phi$')\n",
+ "plt.legend(loc=\"lower right\")\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "G9IwgwKltHz5"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define the model and the loss function"
+ ],
+ "metadata": {
+ "id": "N6VaUq2swa8D"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Model is just a straight line with intercept phi[0] and slope phi[1]\n",
+ "def model(phi,x):\n",
+ " y_pred = phi[0]+phi[1] * x\n",
+ " return y_pred\n",
+ "\n",
+ "# Loss function is 0.5 times sum of squares of residuals for training data\n",
+ "def compute_loss(data_x, data_y, model, phi):\n",
+ " pred_y = model(phi, data_x)\n",
+ " loss = 0.5 * np.sum((pred_y-data_y)*(pred_y-data_y))\n",
+ " return loss"
+ ],
+ "metadata": {
+ "id": "LGHEVUWWiB4f"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Draw the loss function"
+ ],
+ "metadata": {
+ "id": "hr3hs7pKwo0g"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def draw_loss_function(compute_loss, X, y, model, phi_iters):\n",
+ " # Define pretty colormap\n",
+ " my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ " my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ " r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ " g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ " b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ " my_colormap = ListedColormap(np.vstack((r,g,b)).transpose()/255.0)\n",
+ "\n",
+ " # Make grid of intercept/slope values to plot\n",
+ " intercepts_mesh, slopes_mesh = np.meshgrid(np.arange(-1.0,1.0,0.005), np.arange(-1.0,1.0,0.005))\n",
+ " loss_mesh = np.zeros_like(slopes_mesh)\n",
+ " # Compute loss for every set of parameters\n",
+ " for idslope, slope in np.ndenumerate(slopes_mesh):\n",
+ " loss_mesh[idslope] = compute_loss(X, y, model, np.array([[intercepts_mesh[idslope]], [slope]]))\n",
+ "\n",
+ " fig,ax = plt.subplots()\n",
+ " fig.set_size_inches(8,8)\n",
+ " ax.contourf(intercepts_mesh,slopes_mesh,loss_mesh,256,cmap=my_colormap)\n",
+ " ax.contour(intercepts_mesh,slopes_mesh,loss_mesh,40,colors=['#80808080'])\n",
+ " ax.set_ylim([1,-1]); ax.set_xlim([-1,1])\n",
+ "\n",
+ " ax.plot(phi_iters[1,:], phi_iters[0,:],'g-')\n",
+ " ax.set_xlabel('Intercept'); ax.set_ylabel('Slope')\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "UCxa3tZ8a9kz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "draw_loss_function(compute_loss, X[0:1,:], y.T, model, phi_t_all)"
+ ],
+ "metadata": {
+ "id": "pXLLBaSaiI2A"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Draw the evolution of the function"
+ ],
+ "metadata": {
+ "id": "ZsremHW-xFi5"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(X[0:1,:],y.T,'ro')\n",
+ "x_vals = np.arange(0,1,0.001)\n",
+ "ax.plot(x_vals, phi_t_all[0,0]*x_vals + phi_t_all[1,0],'r-', label='t=0.00')\n",
+ "ax.plot(x_vals, phi_t_all[0,10]*x_vals + phi_t_all[1,10],'g-', label='t=0.10')\n",
+ "ax.plot(x_vals, phi_t_all[0,30]*x_vals + phi_t_all[1,30],'b-', label='t=0.30')\n",
+ "ax.plot(x_vals, phi_t_all[0,200]*x_vals + phi_t_all[1,200],'c-', label='t=2.00')\n",
+ "ax.plot(x_vals, phi_t_all[0,1999]*x_vals + phi_t_all[1,1999],'y-', label='t=20.0')\n",
+ "ax.set_xlim([0,1]); ax.set_ylim([-0.5,0.5])\n",
+ "ax.set_xlabel('x'); ax.set_ylabel('y')\n",
+ "plt.legend(loc=\"upper left\")\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "cv9ZrUoRkuhI"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Compute MAP and ML solutions\n",
+ "MLParams = np.linalg.inv(X@X.T)@X@y\n",
+ "sigma_sq_p = 3.0\n",
+ "sigma_sq = 0.05\n",
+ "MAPParams = np.linalg.inv(X@X.T+np.identity(X.shape[0])*sigma_sq/sigma_sq_p)@X@y"
+ ],
+ "metadata": {
+ "id": "OU9oegSOof-o"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Finally, we predict both the mean and the uncertainty in the fitted model as a function of time"
+ ],
+ "metadata": {
+ "id": "Ul__XvOgyYSA"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define x positions to make predictions (appending a 1 to each column)\n",
+ "x_predict = np.arange(0,1,0.01)[None,:]\n",
+ "x_predict = np.concatenate((x_predict,np.ones_like(x_predict)))\n",
+ "nX = x_predict.shape[1]\n",
+ "\n",
+ "# Create variables to store evolution of mean and variance of prediction over time\n",
+ "predict_mean_all = np.zeros((nT,nX))\n",
+ "predict_var_all = np.zeros((nT,nX))\n",
+ "\n",
+ "# Initial covariance\n",
+ "sigma_sq_p = 2.0\n",
+ "cov_init = sigma_sq_p * np.identity(2)\n",
+ "\n",
+ "# Run through each time computing a and b and hence mean and variance of prediction\n",
+ "for t in range(len(t_all)):\n",
+ " a = x_predict.T @(XXTInvX @ (np.identity(3)-expm(-X.T@X * t_all[t])) @ y)\n",
+ " b = x_predict.T -x_predict.T@XXTInvX @ (np.identity(3)-expm(-X.T@X * t_all[t])) @ X.T\n",
+ " predict_mean_all[t:t+1,:] = a.T\n",
+ " predict_cov = b@ cov_init @b.T\n",
+ " # We just want the diagonal of the covariance to plot the uncertainty\n",
+ " predict_var_all[t:t+1,:] = np.reshape(np.diag(predict_cov),(1,nX))"
+ ],
+ "metadata": {
+ "id": "aMPADCuByKWr"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Plot the mean and variance at various times"
+ ],
+ "metadata": {
+ "id": "PZTj93KK7QH6"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def plot_mean_var(X,y,x_predict, predict_mean_all, predict_var_all, this_t, sigma_sq = 0.00001):\n",
+ " fig, ax = plt.subplots()\n",
+ " ax.plot(X[0:1,:],y.T,'ro')\n",
+ " ax.plot(x_predict[0:1,:].T, predict_mean_all[this_t:this_t+1,:].T,'r-')\n",
+ " lower = np.squeeze(predict_mean_all[this_t:this_t+1,:].T-np.sqrt(predict_var_all[this_t:this_t+1,:].T+np.sqrt(sigma_sq)))\n",
+ " upper = np.squeeze(predict_mean_all[this_t:this_t+1,:].T+np.sqrt(predict_var_all[this_t:this_t+1,:].T+np.sqrt(sigma_sq)))\n",
+ " ax.fill_between(np.squeeze(x_predict[0:1,:]), lower, upper, color='lightgray')\n",
+ " ax.set_xlim([0,1]); ax.set_ylim([-0.5,0.5])\n",
+ " ax.set_xlabel('x'); ax.set_ylabel('y')\n",
+ " plt.show()\n",
+ "\n",
+ "plot_mean_var(X,y,x_predict, predict_mean_all, predict_var_all, this_t=0)\n",
+ "plot_mean_var(X,y,x_predict, predict_mean_all, predict_var_all, this_t=40)\n",
+ "plot_mean_var(X,y,x_predict, predict_mean_all, predict_var_all, this_t=80)\n",
+ "plot_mean_var(X,y,x_predict, predict_mean_all, predict_var_all, this_t=200)\n",
+ "plot_mean_var(X,y,x_predict, predict_mean_all, predict_var_all, this_t=500)\n",
+ "plot_mean_var(X,y,x_predict, predict_mean_all, predict_var_all, this_t=1000)"
+ ],
+ "metadata": {
+ "id": "bYAFxgB880-v"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Blogs/BorealisNTK.ipynb b/Blogs/BorealisNTK.ipynb
new file mode 100644
index 0000000..d2062a8
--- /dev/null
+++ b/Blogs/BorealisNTK.ipynb
@@ -0,0 +1,1109 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Neural tangent kernel\n",
+ "\n",
+ "This blog contains code that accompanies the Borealis AI blog on the Neural Tangent Kernel."
+ ],
+ "metadata": {
+ "id": "Ao-2MW3o_SFN"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {
+ "id": "_IQFHZEMZE8T"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from scipy.linalg import expm\n",
+ "from matplotlib import cm\n",
+ "from matplotlib.colors import ListedColormap"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Create some data and draw it."
+ ],
+ "metadata": {
+ "id": "1RYf0uIwA98m"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {
+ "id": "cJNZ2VIcYsD8"
+ },
+ "outputs": [],
+ "source": [
+ "# Input is one dimensional but has ones appended to incorporate biases\n",
+ "X = np.array([[0.2, 0.4, 0.8],[1,1,1]])\n",
+ "y = np.array([[-0.05],[-0.2],[0.3]])\n",
+ "\n",
+ "D_in = X.shape[0] # No of input dimensions (2)\n",
+ "I = X.shape[1] # No of data points (3)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 449
+ },
+ "id": "FpFlD4nUZDRt",
+ "outputId": "b0cd7f72-4f63-4e19-ab3c-b6b274bbbad1"
+ },
+ "outputs": [
+ {
+ "output_type": "display_data",
+ "data": {
+ "text/plain": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "\n",
+ "# **CM20315 Background Mathematics**\n",
+ "\n",
+ "The purpose of this Python notebook is to make sure you can use CoLab and to familiarize yourself with some of the background mathematical concepts that you are going to need to understand deep learning. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Convolution I \n",
+ "\n",
+ "This notebook investigates the convolution operation. It asks you to hand code a convolution so we can be sure that we are computing the same thing as in PyTorch. The subsequent notebooks use the convolutional layers in PyTorch directly."
+ ],
+ "metadata": {
+ "id": "VB_crnDGASX-"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import torch\n",
+ "# Set to print in reasonable form\n",
+ "np.set_printoptions(precision=3, floatmode=\"fixed\")\n",
+ "torch.set_printoptions(precision=3)"
+ ],
+ "metadata": {
+ "id": "YAoWDUb_DezG"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "This routine performs convolution in PyTorch"
+ ],
+ "metadata": {
+ "id": "eAwYWXzAElHG"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Perform convolution in PyTorch\n",
+ "def conv_pytorch(image, conv_weights, stride=1, pad =1):\n",
+ " # Convert image and kernel to tensors\n",
+ " image_tensor = torch.from_numpy(image) # (batchSize, channelsIn, imageHeightIn, =imageWidthIn)\n",
+ " conv_weights_tensor = torch.from_numpy(conv_weights) # (channelsOut, channelsIn, kernelHeight, kernelWidth) \n",
+ " # Do the convolution\n",
+ " output_tensor = torch.nn.functional.conv2d(image_tensor, conv_weights_tensor, stride=stride, padding=pad) \n",
+ " # Convert back from PyTorch and return\n",
+ " return(output_tensor.numpy()) # (batchSize channelsOut imageHeightOut imageHeightIn)"
+ ],
+ "metadata": {
+ "id": "xsmUIN-3BlWr"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "First we'll start with the simplest 2D convolution. Just one channel in and one channel out. A single image in the batch."
+ ],
+ "metadata": {
+ "id": "A3Sm8bUWtDNO"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Perform convolution in numpy\n",
+ "def conv_numpy_1(image, weights, pad=1):\n",
+ " \n",
+ " # Perform zero padding \n",
+ " if pad != 0:\n",
+ " image = np.pad(image, ((0, 0), (0 ,0), (pad, pad), (pad, pad)),'constant')\n",
+ " \n",
+ " # Get sizes of image array and kernel weights\n",
+ " batchSize, channelsIn, imageHeightIn, imageWidthIn = image.shape\n",
+ " channelsOut, channelsIn, kernelHeight, kernelWidth = weights.shape\n",
+ "\n",
+ " # Get size of output arrays\n",
+ " imageHeightOut = np.floor(1 + imageHeightIn - kernelHeight).astype(int)\n",
+ " imageWidthOut = np.floor(1 + imageWidthIn - kernelWidth).astype(int)\n",
+ "\n",
+ " # Create output\n",
+ " out = np.zeros((batchSize, channelsOut, imageHeightOut, imageWidthOut), dtype=np.float32) \n",
+ " \n",
+ " for c_y in range(imageHeightOut):\n",
+ " for c_x in range(imageWidthOut):\n",
+ " for c_kernel_y in range(kernelHeight):\n",
+ " for c_kernel_x in range(kernelWidth):\n",
+ " # TODO -- Retrieve the image pixel and the weight from the convolution\n",
+ " # Only one image in batch, one input channel and one output channel, so these indices should all be zero\n",
+ " # Replace the two lines below\n",
+ " this_pixel_value = 1.0\n",
+ " this_weight = 1.0\n",
+ " \n",
+ " # Multiply these together and add to the output at this position\n",
+ " out[0, 0, c_y, c_x] += np.sum(this_pixel_value * this_weight) \n",
+ " \n",
+ " return out"
+ ],
+ "metadata": {
+ "id": "EF8FWONVLo1Q"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set random seed so we always get same answer\n",
+ "np.random.seed(1) \n",
+ "n_batch = 1\n",
+ "image_height = 4\n",
+ "image_width = 6\n",
+ "channels_in = 1\n",
+ "kernel_size = 3\n",
+ "channels_out = 1\n",
+ "\n",
+ "# Create random input image\n",
+ "input_image= np.random.normal(size=(n_batch, channels_in, image_height, image_width))\n",
+ "# Create random convolution kernel weights\n",
+ "conv_weights = np.random.normal(size=(channels_out, channels_in, kernel_size, kernel_size))\n",
+ "\n",
+ "# Perform convolution using PyTorch\n",
+ "conv_results_pytorch = conv_pytorch(input_image, conv_weights, stride=1, pad=1)\n",
+ "print(\"PyTorch Results\")\n",
+ "print(conv_results_pytorch)\n",
+ "\n",
+ "# Perform convolution in numpy\n",
+ "print(\"Your results\")\n",
+ "conv_results_numpy = conv_numpy_1(input_image, conv_weights)\n",
+ "print(conv_results_numpy)"
+ ],
+ "metadata": {
+ "id": "iw9KqXZTHN8v"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's now add in the possibility of using different strides"
+ ],
+ "metadata": {
+ "id": "IYj_lxeGzaHX"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Perform convolution in numpy\n",
+ "def conv_numpy_2(image, weights, stride=1, pad=1):\n",
+ " \n",
+ " # Perform zero padding \n",
+ " if pad != 0:\n",
+ " image = np.pad(image, ((0, 0), (0 ,0), (pad, pad), (pad, pad)),'constant')\n",
+ " \n",
+ " # Get sizes of image array and kernel weights\n",
+ " batchSize, channelsIn, imageHeightIn, imageWidthIn = image.shape\n",
+ " channelsOut, channelsIn, kernelHeight, kernelWidth = weights.shape\n",
+ "\n",
+ " # Get size of output arrays\n",
+ " imageHeightOut = np.floor(1 + (imageHeightIn - kernelHeight) / stride).astype(int)\n",
+ " imageWidthOut = np.floor(1 + (imageWidthIn - kernelWidth) / stride).astype(int)\n",
+ " \n",
+ " # Create output\n",
+ " out = np.zeros((batchSize, channelsOut, imageHeightOut, imageWidthOut), dtype=np.float32) \n",
+ " \n",
+ " for c_y in range(imageHeightOut):\n",
+ " for c_x in range(imageWidthOut):\n",
+ " for c_kernel_y in range(kernelHeight):\n",
+ " for c_kernel_x in range(kernelWidth):\n",
+ " # TODO -- Retrieve the image pixel and the weight from the convolution\n",
+ " # Only one image in batch, one input channel and one output channel, so these indices should all be zero\n",
+ " # Replace the two lines below\n",
+ " this_pixel_value = 1.0\n",
+ " this_weight = 1.0\n",
+ "\n",
+ " # Multiply these together and add to the output at this position\n",
+ " out[0, 0, c_y, c_x] += np.sum(this_pixel_value * this_weight) \n",
+ " \n",
+ " return out"
+ ],
+ "metadata": {
+ "id": "GiujmLhqHN1F"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set random seed so we always get same answer\n",
+ "np.random.seed(1) \n",
+ "n_batch = 1\n",
+ "image_height = 12\n",
+ "image_width = 10\n",
+ "channels_in = 1\n",
+ "kernel_size = 3\n",
+ "channels_out = 1\n",
+ "stride = 2\n",
+ "\n",
+ "# Create random input image\n",
+ "input_image= np.random.normal(size=(n_batch, channels_in, image_height, image_width))\n",
+ "# Create random convolution kernel weights\n",
+ "conv_weights = np.random.normal(size=(channels_out, channels_in, kernel_size, kernel_size))\n",
+ "\n",
+ "# Perform convolution using PyTorch\n",
+ "conv_results_pytorch = conv_pytorch(input_image, conv_weights, stride, pad=1)\n",
+ "print(\"PyTorch Results\")\n",
+ "print(conv_results_pytorch)\n",
+ "\n",
+ "# Perform convolution in numpy\n",
+ "print(\"Your results\")\n",
+ "conv_results_numpy = conv_numpy_2(input_image, conv_weights, stride, pad=1)\n",
+ "print(conv_results_numpy)"
+ ],
+ "metadata": {
+ "id": "FeJy6Bvozgxq"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now we'll introduce multiple input and output channels"
+ ],
+ "metadata": {
+ "id": "3flq1Wan2gX-"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Perform convolution in numpy\n",
+ "def conv_numpy_3(image, weights, stride=1, pad=1):\n",
+ " \n",
+ " # Perform zero padding \n",
+ " if pad != 0:\n",
+ " image = np.pad(image, ((0, 0), (0 ,0), (pad, pad), (pad, pad)),'constant')\n",
+ " \n",
+ " # Get sizes of image array and kernel weights\n",
+ " batchSize, channelsIn, imageHeightIn, imageWidthIn = image.shape\n",
+ " channelsOut, channelsIn, kernelHeight, kernelWidth = weights.shape\n",
+ "\n",
+ " # Get size of output arrays\n",
+ " imageHeightOut = np.floor(1 + (imageHeightIn - kernelHeight) / stride).astype(int)\n",
+ " imageWidthOut = np.floor(1 + (imageWidthIn - kernelWidth) / stride).astype(int)\n",
+ " \n",
+ " # Create output\n",
+ " out = np.zeros((batchSize, channelsOut, imageHeightOut, imageWidthOut), dtype=np.float32) \n",
+ " \n",
+ " for c_y in range(imageHeightOut):\n",
+ " for c_x in range(imageWidthOut):\n",
+ " for c_channel_out in range(channelsOut):\n",
+ " for c_channel_in in range(channelsIn):\n",
+ " for c_kernel_y in range(kernelHeight):\n",
+ " for c_kernel_x in range(kernelWidth):\n",
+ " # TODO -- Retrieve the image pixel and the weight from the convolution\n",
+ " # Only one image in batch so this index should be zero\n",
+ " # Replace the two lines below\n",
+ " this_pixel_value = 1.0\n",
+ " this_weight = 1.0\n",
+ "\n",
+ " # Multiply these together and add to the output at this position\n",
+ " out[0, c_channel_out, c_y, c_x] += np.sum(this_pixel_value * this_weight) \n",
+ " return out"
+ ],
+ "metadata": {
+ "id": "AvdRWGiU2ppX"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set random seed so we always get same answer\n",
+ "np.random.seed(1) \n",
+ "n_batch = 1\n",
+ "image_height = 4\n",
+ "image_width = 6\n",
+ "channels_in = 5\n",
+ "kernel_size = 3\n",
+ "channels_out = 2\n",
+ "\n",
+ "# Create random input image\n",
+ "input_image= np.random.normal(size=(n_batch, channels_in, image_height, image_width))\n",
+ "# Create random convolution kernel weights\n",
+ "conv_weights = np.random.normal(size=(channels_out, channels_in, kernel_size, kernel_size))\n",
+ "\n",
+ "# Perform convolution using PyTorch\n",
+ "conv_results_pytorch = conv_pytorch(input_image, conv_weights, stride=1, pad=1)\n",
+ "print(\"PyTorch Results\")\n",
+ "print(conv_results_pytorch)\n",
+ "\n",
+ "# Perform convolution in numpy\n",
+ "print(\"Your results\")\n",
+ "conv_results_numpy = conv_numpy_3(input_image, conv_weights, stride=1, pad=1)\n",
+ "print(conv_results_numpy)"
+ ],
+ "metadata": {
+ "id": "mdSmjfvY4li2"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now we'll do the full convolution with multiple images (batch size > 1), and multiple input channels, multiple output channels."
+ ],
+ "metadata": {
+ "id": "Q2MUFebdsJbH"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Perform convolution in numpy\n",
+ "def conv_numpy_4(image, weights, stride=1, pad=1):\n",
+ " \n",
+ " # Perform zero padding \n",
+ " if pad != 0:\n",
+ " image = np.pad(image, ((0, 0), (0 ,0), (pad, pad), (pad, pad)),'constant')\n",
+ " \n",
+ " # Get sizes of image array and kernel weights\n",
+ " batchSize, channelsIn, imageHeightIn, imageWidthIn = image.shape\n",
+ " channelsOut, channelsIn, kernelHeight, kernelWidth = weights.shape\n",
+ "\n",
+ " # Get size of output arrays\n",
+ " imageHeightOut = np.floor(1 + (imageHeightIn - kernelHeight) / stride).astype(int)\n",
+ " imageWidthOut = np.floor(1 + (imageWidthIn - kernelWidth) / stride).astype(int)\n",
+ " \n",
+ " # Create output\n",
+ " out = np.zeros((batchSize, channelsOut, imageHeightOut, imageWidthOut), dtype=np.float32) \n",
+ " \n",
+ " for c_batch in range(batchSize):\n",
+ " for c_y in range(imageHeightOut):\n",
+ " for c_x in range(imageWidthOut):\n",
+ " for c_channel_out in range(channelsOut):\n",
+ " for c_channel_in in range(channelsIn):\n",
+ " for c_kernel_y in range(kernelHeight):\n",
+ " for c_kernel_x in range(kernelWidth):\n",
+ " # TODO -- Retrieve the image pixel and the weight from the convolution\n",
+ " # Replace the two lines below\n",
+ " this_pixel_value = 1.0\n",
+ " this_weight = 1.0\n",
+ " \n",
+ " # Multiply these together and add to the output at this position\n",
+ " out[c_batch, c_channel_out, c_y, c_x] += np.sum(this_pixel_value * this_weight) \n",
+ " return out"
+ ],
+ "metadata": {
+ "id": "5WePF-Y-sC1y"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "1w2GEBtqAM2P"
+ },
+ "outputs": [],
+ "source": [
+ "# Set random seed so we always get same answer\n",
+ "np.random.seed(1) \n",
+ "n_batch = 2\n",
+ "image_height = 4\n",
+ "image_width = 6\n",
+ "channels_in = 5\n",
+ "kernel_size = 3\n",
+ "channels_out = 2\n",
+ "\n",
+ "# Create random input image\n",
+ "input_image= np.random.normal(size=(n_batch, channels_in, image_height, image_width))\n",
+ "# Create random convolution kernel weights\n",
+ "conv_weights = np.random.normal(size=(channels_out, channels_in, kernel_size, kernel_size))\n",
+ "\n",
+ "# Perform convolution using PyTorch\n",
+ "conv_results_pytorch = conv_pytorch(input_image, conv_weights, stride=1, pad=1)\n",
+ "print(\"PyTorch Results\")\n",
+ "print(conv_results_pytorch)\n",
+ "\n",
+ "# Perform convolution in numpy\n",
+ "print(\"Your results\")\n",
+ "conv_results_numpy = conv_numpy_4(input_image, conv_weights, stride=1, pad=1)\n",
+ "print(conv_results_numpy)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "Lody75JB5By7"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
diff --git a/CM20315/CM20315_Convolution_II.ipynb b/CM20315/CM20315_Convolution_II.ipynb
new file mode 100644
index 0000000..4d87123
--- /dev/null
+++ b/CM20315/CM20315_Convolution_II.ipynb
@@ -0,0 +1,253 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyN4fpyg0d75XccLLsNahur1",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Convolution II -- MNIST1D\n",
+ "\n",
+ "This notebook investigates what happens when we use convolutional layers instead of fully-connected layers for the MNIST-1D from the coursework.\n",
+ "\n",
+ "We'll build the network from figure 10.7 in the notes.\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import os\n",
+ "import torch, torch.nn as nn\n",
+ "from torch.utils.data import TensorDataset, DataLoader\n",
+ "from torch.optim.lr_scheduler import StepLR\n",
+ "import matplotlib.pyplot as plt\n",
+ "import random"
+ ],
+ "metadata": {
+ "id": "YrXWAH7sUWvU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run this once to copy the train and validation data to your CoLab environment \n",
+ "# or download from my github to your local machine if you are doing this locally\n",
+ "if not os.path.exists('./train_data_x.npy'):\n",
+ " !wget https://github.com/udlbook/udlbook/raw/main/practicals/train_data_x.npy\n",
+ " !wget https://github.com/udlbook/udlbook/raw/main/practicals/train_data_y.npy\n",
+ " !wget https://github.com/udlbook/udlbook/raw/main/practicals/val_data_x.npy\n",
+ " !wget https://github.com/udlbook/udlbook/raw/main/practicals/val_data_y.npy "
+ ],
+ "metadata": {
+ "id": "wScBGXXFVadm"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Load in the data\n",
+ "train_data_x = np.load('train_data_x.npy')\n",
+ "train_data_y = np.load('train_data_y.npy')\n",
+ "val_data_x = np.load('val_data_x.npy')\n",
+ "val_data_y = np.load('val_data_y.npy')\n",
+ "# Print out sizes\n",
+ "print(\"Train data: %d examples (columns), each of which has %d dimensions (rows)\"%((train_data_x.shape[1],train_data_x.shape[0])))\n",
+ "print(\"Validation data: %d examples (columns), each of which has %d dimensions (rows)\"%((val_data_x.shape[1],val_data_x.shape[0])))"
+ ],
+ "metadata": {
+ "id": "8bKADvLHbiV5"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define the network"
+ ],
+ "metadata": {
+ "id": "_sFvRDGrl4qe"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "\n",
+ "# TODO Create a model with the following layers\n",
+ "# 1. Convolutional layer, (input=length 40 and 1 channel, kernel size 3x3, stride 2, padding=\"valid\", 15 output channels ) \n",
+ "# 2. ReLU\n",
+ "# 3. Convolutional layer, (input=length 19 and 15 channels, kernel size 3x3, stride 2, padding=\"valid\", 15 output channels )\n",
+ "# 4. ReLU\n",
+ "# 5. Convolutional layer, (input=length 9 and 15 channels, kernel size 3x3, stride 2, padding=\"valid\", 15 output channels)\n",
+ "# 6. ReLU\n",
+ "# 7. Flatten (converts 4x15) to length 60\n",
+ "# 8. Linear layer (input size = 60, output size = 10)\n",
+ "# References:\n",
+ "# https://pytorch.org/docs/1.13/generated/torch.nn.Conv1d.html?highlight=conv1d#torch.nn.Conv1d\n",
+ "# https://pytorch.org/docs/stable/generated/torch.nn.Flatten.html\n",
+ "# https://pytorch.org/docs/1.13/generated/torch.nn.Linear.html?highlight=linear#torch.nn.Linear\n",
+ "\n",
+ "# Replace the following function which just runs a standard fully connected network\n",
+ "# The flatten at the beginning is because we are passing in the data in a slightly different format.\n",
+ "model = nn.Sequential(\n",
+ "nn.Flatten(),\n",
+ "nn.Linear(40, 100),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(100, 100),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(100, 10))"
+ ],
+ "metadata": {
+ "id": "FslroPJJffrh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# He initialization of weights\n",
+ "def weights_init(layer_in):\n",
+ " if isinstance(layer_in, nn.Linear):\n",
+ " nn.init.kaiming_uniform_(layer_in.weight)\n",
+ " layer_in.bias.data.fill_(0.0)"
+ ],
+ "metadata": {
+ "id": "YgLaex1pfhqz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# You need all this stuff to ensure that PyTorch is deterministic\n",
+ "def set_seed(seed):\n",
+ " torch.manual_seed(seed)\n",
+ " torch.cuda.manual_seed_all(seed)\n",
+ " torch.backends.cudnn.deterministic = True\n",
+ " torch.backends.cudnn.benchmark = False\n",
+ " np.random.seed(seed)\n",
+ " random.seed(seed)\n",
+ " os.environ['PYTHONHASHSEED'] = str(seed)"
+ ],
+ "metadata": {
+ "id": "zXRmxCQNnL_M"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set seed so always get same result (do not change)\n",
+ "set_seed(1)\n",
+ "\n",
+ "# choose cross entropy loss function (equation 5.24 in the loss notes)\n",
+ "loss_function = nn.CrossEntropyLoss()\n",
+ "# construct SGD optimizer and initialize learning rate and momentum\n",
+ "optimizer = torch.optim.SGD(model.parameters(), lr = 0.05, momentum=0.9)\n",
+ "# object that decreases learning rate by half every 10 epochs\n",
+ "scheduler = StepLR(optimizer, step_size=10, gamma=0.5)\n",
+ "# create 100 dummy data points and store in data loader class\n",
+ "x_train = torch.tensor(train_data_x.transpose().astype('float32'))\n",
+ "y_train = torch.tensor(train_data_y.astype('long'))\n",
+ "x_val= torch.tensor(val_data_x.transpose().astype('float32'))\n",
+ "y_val = torch.tensor(val_data_y.astype('long'))\n",
+ "\n",
+ "# load the data into a class that creates the batches\n",
+ "data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
+ "\n",
+ "# Initialize model weights\n",
+ "model.apply(weights_init)\n",
+ "\n",
+ "# loop over the dataset n_epoch times\n",
+ "n_epoch = 50\n",
+ "# store the loss and the % correct at each epoch\n",
+ "losses_train = np.zeros((n_epoch))\n",
+ "errors_train = np.zeros((n_epoch))\n",
+ "losses_val = np.zeros((n_epoch))\n",
+ "errors_val = np.zeros((n_epoch))\n",
+ "\n",
+ "for epoch in range(n_epoch):\n",
+ " # loop over batches\n",
+ " for i, data in enumerate(data_loader):\n",
+ " # retrieve inputs and labels for this batch\n",
+ " x_batch, y_batch = data\n",
+ " # zero the parameter gradients\n",
+ " optimizer.zero_grad()\n",
+ " # forward pass -- calculate model output\n",
+ " pred = model(x_batch[:,None,:])\n",
+ " # compute the loss\n",
+ " loss = loss_function(pred, y_batch)\n",
+ " # backward pass\n",
+ " loss.backward()\n",
+ " # SGD update\n",
+ " optimizer.step()\n",
+ "\n",
+ " # Run whole dataset to get statistics -- normally wouldn't do this\n",
+ " pred_train = model(x_train[:,None,:])\n",
+ " pred_val = model(x_val[:,None,:])\n",
+ " _, predicted_train_class = torch.max(pred_train.data, 1)\n",
+ " _, predicted_val_class = torch.max(pred_val.data, 1)\n",
+ " errors_train[epoch] = 100 - 100 * (predicted_train_class == y_train).float().sum() / len(y_train)\n",
+ " errors_val[epoch]= 100 - 100 * (predicted_val_class == y_val).float().sum() / len(y_val)\n",
+ " losses_train[epoch] = loss_function(pred_train, y_train).item()\n",
+ " losses_val[epoch]= loss_function(pred_val, y_val).item()\n",
+ " print(f'Epoch {epoch:5d}, train loss {losses_train[epoch]:.6f}, train error {errors_train[epoch]:3.2f}, val loss {losses_val[epoch]:.6f}, percent error {errors_val[epoch]:3.2f}')\n",
+ " \n",
+ " # tell scheduler to consider updating learning rate\n",
+ " scheduler.step()\n",
+ "\n",
+ "# Plot the results\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(errors_train,'r-',label='train')\n",
+ "ax.plot(errors_val,'b-',label='validation')\n",
+ "ax.set_ylim(0,100); ax.set_xlim(0,n_epoch)\n",
+ "ax.set_xlabel('Epoch'); ax.set_ylabel('Error')\n",
+ "ax.set_title('Part I: Validation Result %3.2f'%(errors_val[-1]))\n",
+ "ax.legend()\n",
+ "ax.plot([0,n_epoch],[37.45, 37.45],'k:') # Original results. You should be better than this!\n",
+ "plt.savefig('Coursework_I_Results.png',format='png')\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "NYw8I_3mmX5c"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
diff --git a/CM20315/CM20315_Convolution_III.ipynb b/CM20315/CM20315_Convolution_III.ipynb
new file mode 100644
index 0000000..ce0039f
--- /dev/null
+++ b/CM20315/CM20315_Convolution_III.ipynb
@@ -0,0 +1,648 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyM4FAP7pqe8LpKfmixZN07G",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Convolution III -- MNIST\n",
+ "\n",
+ "This notebook builds a proper network for 2D convolution. It works with the MNIST dataset, which was the original classic dataset for classifying images. The network will take a 28x28 grayscale image and classify it into one of 10 classes representing a digit.\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import torch\n",
+ "import torchvision\n",
+ "import torch.nn as nn\n",
+ "import torch.nn.functional as F\n",
+ "import torch.optim as optim\n",
+ "import matplotlib.pyplot as plt\n",
+ "import random"
+ ],
+ "metadata": {
+ "id": "YrXWAH7sUWvU"
+ },
+ "execution_count": 1,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run this once to load the train and test data straight into a dataloader class\n",
+ "# that will provide the batches\n",
+ "batch_size_train = 64\n",
+ "batch_size_test = 1000\n",
+ "train_loader = torch.utils.data.DataLoader(\n",
+ " torchvision.datasets.MNIST('/files/', train=True, download=True,\n",
+ " transform=torchvision.transforms.Compose([\n",
+ " torchvision.transforms.ToTensor(),\n",
+ " torchvision.transforms.Normalize(\n",
+ " (0.1307,), (0.3081,))\n",
+ " ])),\n",
+ " batch_size=batch_size_train, shuffle=True)\n",
+ "\n",
+ "test_loader = torch.utils.data.DataLoader(\n",
+ " torchvision.datasets.MNIST('/files/', train=False, download=True,\n",
+ " transform=torchvision.transforms.Compose([\n",
+ " torchvision.transforms.ToTensor(),\n",
+ " torchvision.transforms.Normalize(\n",
+ " (0.1307,), (0.3081,))\n",
+ " ])),\n",
+ " batch_size=batch_size_test, shuffle=True)"
+ ],
+ "metadata": {
+ "id": "wScBGXXFVadm"
+ },
+ "execution_count": 2,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's draw some of the training data\n",
+ "examples = enumerate(test_loader)\n",
+ "batch_idx, (example_data, example_targets) = next(examples)\n",
+ "\n",
+ "fig = plt.figure()\n",
+ "for i in range(6):\n",
+ " plt.subplot(2,3,i+1)\n",
+ " plt.tight_layout()\n",
+ " plt.imshow(example_data[i][0], cmap='gray', interpolation='none')\n",
+ " plt.title(\"Ground Truth: {}\".format(example_targets[i]))\n",
+ " plt.xticks([])\n",
+ " plt.yticks([])\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "8bKADvLHbiV5",
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 284
+ },
+ "outputId": "ec39acb8-05e0-48ae-cb6e-1b8578d00e68"
+ },
+ "execution_count": 3,
+ "outputs": [
+ {
+ "output_type": "display_data",
+ "data": {
+ "text/plain": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Coursework I -- Model hyperparameters\n",
+ "\n",
+ "The goal of the coursework is to modify a simple bit of numpy code that trains a network and measures the performance on a validation set for the MNist 1D dataset. \n",
+ "\n",
+ "In this coursework, you need to modify the **model hyperparameters** (only) to improve the performance over the current attempt. This could mean the number of layers, the number of hidden units per layer, or the type of activation function, or any combination of the three. \n",
+ "\n",
+ "The only constraint is that you MUST use a fully connected network (no convolutional networks for now if you have read ahead in the book).\n",
+ "\n",
+ "You don't have to improve the performance much. A few tenths of a percent is fine. It just has to be better to get full marks.\n",
+ "\n",
+ "You will need to upload three things to Moodle:\n",
+ "1. The image that this notebook saves (click the folder icon on the left on colab to download it)\n",
+ "2. The lines of code you changed\n",
+ "3. The whole notebook as a .ipynb file. You can do this on the File menu\n",
+ "\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import os\n",
+ "import torch, torch.nn as nn\n",
+ "from torch.utils.data import TensorDataset, DataLoader\n",
+ "from torch.optim.lr_scheduler import StepLR\n",
+ "import matplotlib.pyplot as plt\n",
+ "import random"
+ ],
+ "metadata": {
+ "id": "YrXWAH7sUWvU"
+ },
+ "execution_count": 1,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run this once to copy the train and validation data to your CoLab environment \n",
+ "# or download from my github to your local machine if you are doing this locally\n",
+ "if not os.path.exists('./train_data_x.npy'):\n",
+ " !wget https://github.com/udlbook/udlbook/raw/main/practicals/train_data_x.npy\n",
+ " !wget https://github.com/udlbook/udlbook/raw/main/practicals/train_data_y.npy\n",
+ " !wget https://github.com/udlbook/udlbook/raw/main/practicals/val_data_x.npy\n",
+ " !wget https://github.com/udlbook/udlbook/raw/main/practicals/val_data_y.npy "
+ ],
+ "metadata": {
+ "id": "wScBGXXFVadm",
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "outputId": "970c192f-33ad-45ee-dc12-b1b9a30b50d7"
+ },
+ "execution_count": 2,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "--2022-11-21 12:37:03-- https://github.com/udlbook/udlbook/raw/main/practicals/train_data_x.npy\n",
+ "Resolving github.com (github.com)... 140.82.114.3\n",
+ "Connecting to github.com (github.com)|140.82.114.3|:443... connected.\n",
+ "HTTP request sent, awaiting response... 302 Found\n",
+ "Location: https://raw.githubusercontent.com/udlbook/udlbook/main/practicals/train_data_x.npy [following]\n",
+ "--2022-11-21 12:37:04-- https://raw.githubusercontent.com/udlbook/udlbook/main/practicals/train_data_x.npy\n",
+ "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
+ "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
+ "HTTP request sent, awaiting response... 200 OK\n",
+ "Length: 1280128 (1.2M) [application/octet-stream]\n",
+ "Saving to: ‘train_data_x.npy’\n",
+ "\n",
+ "train_data_x.npy 100%[===================>] 1.22M --.-KB/s in 0.05s \n",
+ "\n",
+ "2022-11-21 12:37:04 (22.9 MB/s) - ‘train_data_x.npy’ saved [1280128/1280128]\n",
+ "\n",
+ "--2022-11-21 12:37:04-- https://github.com/udlbook/udlbook/raw/main/practicals/train_data_y.npy\n",
+ "Resolving github.com (github.com)... 140.82.112.3\n",
+ "Connecting to github.com (github.com)|140.82.112.3|:443... connected.\n",
+ "HTTP request sent, awaiting response... 302 Found\n",
+ "Location: https://raw.githubusercontent.com/udlbook/udlbook/main/practicals/train_data_y.npy [following]\n",
+ "--2022-11-21 12:37:04-- https://raw.githubusercontent.com/udlbook/udlbook/main/practicals/train_data_y.npy\n",
+ "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
+ "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
+ "HTTP request sent, awaiting response... 200 OK\n",
+ "Length: 32128 (31K) [application/octet-stream]\n",
+ "Saving to: ‘train_data_y.npy’\n",
+ "\n",
+ "train_data_y.npy 100%[===================>] 31.38K --.-KB/s in 0.002s \n",
+ "\n",
+ "2022-11-21 12:37:04 (13.9 MB/s) - ‘train_data_y.npy’ saved [32128/32128]\n",
+ "\n",
+ "--2022-11-21 12:37:04-- https://github.com/udlbook/udlbook/raw/main/practicals/val_data_x.npy\n",
+ "Resolving github.com (github.com)... 140.82.113.3\n",
+ "Connecting to github.com (github.com)|140.82.113.3|:443... connected.\n",
+ "HTTP request sent, awaiting response... 302 Found\n",
+ "Location: https://raw.githubusercontent.com/udlbook/udlbook/main/practicals/val_data_x.npy [following]\n",
+ "--2022-11-21 12:37:04-- https://raw.githubusercontent.com/udlbook/udlbook/main/practicals/val_data_x.npy\n",
+ "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
+ "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
+ "HTTP request sent, awaiting response... 200 OK\n",
+ "Length: 640128 (625K) [application/octet-stream]\n",
+ "Saving to: ‘val_data_x.npy’\n",
+ "\n",
+ "val_data_x.npy 100%[===================>] 625.12K --.-KB/s in 0.04s \n",
+ "\n",
+ "2022-11-21 12:37:05 (14.1 MB/s) - ‘val_data_x.npy’ saved [640128/640128]\n",
+ "\n",
+ "--2022-11-21 12:37:05-- https://github.com/udlbook/udlbook/raw/main/practicals/val_data_y.npy\n",
+ "Resolving github.com (github.com)... 140.82.114.4\n",
+ "Connecting to github.com (github.com)|140.82.114.4|:443... connected.\n",
+ "HTTP request sent, awaiting response... 302 Found\n",
+ "Location: https://raw.githubusercontent.com/udlbook/udlbook/main/practicals/val_data_y.npy [following]\n",
+ "--2022-11-21 12:37:05-- https://raw.githubusercontent.com/udlbook/udlbook/main/practicals/val_data_y.npy\n",
+ "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
+ "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
+ "HTTP request sent, awaiting response... 200 OK\n",
+ "Length: 16128 (16K) [application/octet-stream]\n",
+ "Saving to: ‘val_data_y.npy’\n",
+ "\n",
+ "val_data_y.npy 100%[===================>] 15.75K --.-KB/s in 0s \n",
+ "\n",
+ "2022-11-21 12:37:05 (31.0 MB/s) - ‘val_data_y.npy’ saved [16128/16128]\n",
+ "\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Load in the data\n",
+ "train_data_x = np.load('train_data_x.npy')\n",
+ "train_data_y = np.load('train_data_y.npy')\n",
+ "val_data_x = np.load('val_data_x.npy')\n",
+ "val_data_y = np.load('val_data_y.npy')\n",
+ "# Print out sizes\n",
+ "print(\"Train data: %d examples (columns), each of which has %d dimensions (rows)\"%((train_data_x.shape[1],train_data_x.shape[0])))\n",
+ "print(\"Validation data: %d examples (columns), each of which has %d dimensions (rows)\"%((val_data_x.shape[1],val_data_x.shape[0])))"
+ ],
+ "metadata": {
+ "id": "8bKADvLHbiV5",
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "outputId": "869f360f-3f9b-4e3a-f8c6-cd69fb331709"
+ },
+ "execution_count": 3,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Train data: 4000 examples (columns), each of which has 40 dimensions (rows)\n",
+ "Validation data: 2000 examples (columns), each of which has 40 dimensions (rows)\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define the network"
+ ],
+ "metadata": {
+ "id": "_sFvRDGrl4qe"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# YOU SHOULD ONLY CHANGE THIS CELL!\n",
+ "\n",
+ "# There are 40 input dimensions and 10 output dimensions for this data\n",
+ "# The inputs correspond to the 40 offsets in the MNIST1D template.\n",
+ "D_i = 40\n",
+ "# The outputs correspond to the 10 digits\n",
+ "D_o = 10\n",
+ "\n",
+ "# Number of hidden units in layers 1 and 2\n",
+ "D_1 = 100\n",
+ "D_2 = 100\n",
+ "\n",
+ "# create model with two hidden layers\n",
+ "model = nn.Sequential(\n",
+ "nn.Linear(D_i, D_1),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(D_1, D_2),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(D_2, D_o))"
+ ],
+ "metadata": {
+ "id": "FslroPJJffrh"
+ },
+ "execution_count": 4,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# He initialization of weights\n",
+ "def weights_init(layer_in):\n",
+ " if isinstance(layer_in, nn.Linear):\n",
+ " nn.init.kaiming_uniform_(layer_in.weight)\n",
+ " layer_in.bias.data.fill_(0.0)"
+ ],
+ "metadata": {
+ "id": "YgLaex1pfhqz"
+ },
+ "execution_count": 5,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# You need all this stuff to ensure that PyTorch is deterministic\n",
+ "def set_seed(seed):\n",
+ " torch.manual_seed(seed)\n",
+ " torch.cuda.manual_seed_all(seed)\n",
+ " torch.backends.cudnn.deterministic = True\n",
+ " torch.backends.cudnn.benchmark = False\n",
+ " np.random.seed(seed)\n",
+ " random.seed(seed)\n",
+ " os.environ['PYTHONHASHSEED'] = str(seed)"
+ ],
+ "metadata": {
+ "id": "zXRmxCQNnL_M"
+ },
+ "execution_count": 6,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set seed so always get same result (do not change)\n",
+ "set_seed(1)\n",
+ "\n",
+ "# choose cross entropy loss function (equation 5.24 in the loss notes)\n",
+ "loss_function = nn.CrossEntropyLoss()\n",
+ "# construct SGD optimizer and initialize learning rate and momentum\n",
+ "optimizer = torch.optim.SGD(model.parameters(), lr = 0.05, momentum=0.9)\n",
+ "# object that decreases learning rate by half every 10 epochs\n",
+ "scheduler = StepLR(optimizer, step_size=10, gamma=0.5)\n",
+ "# create 100 dummy data points and store in data loader class\n",
+ "x_train = torch.tensor(train_data_x.transpose().astype('float32'))\n",
+ "y_train = torch.tensor(train_data_y.astype('long'))\n",
+ "x_val= torch.tensor(val_data_x.transpose().astype('float32'))\n",
+ "y_val = torch.tensor(val_data_y.astype('long'))\n",
+ "\n",
+ "# load the data into a class that creates the batches\n",
+ "data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
+ "\n",
+ "# Initialize model weights\n",
+ "model.apply(weights_init)\n",
+ "\n",
+ "# loop over the dataset n_epoch times\n",
+ "n_epoch = 50\n",
+ "# store the loss and the % correct at each epoch\n",
+ "losses_train = np.zeros((n_epoch))\n",
+ "errors_train = np.zeros((n_epoch))\n",
+ "losses_val = np.zeros((n_epoch))\n",
+ "errors_val = np.zeros((n_epoch))\n",
+ "\n",
+ "for epoch in range(n_epoch):\n",
+ " # loop over batches\n",
+ " for i, data in enumerate(data_loader):\n",
+ " # retrieve inputs and labels for this batch\n",
+ " x_batch, y_batch = data\n",
+ " # zero the parameter gradients\n",
+ " optimizer.zero_grad()\n",
+ " # forward pass -- calculate model output\n",
+ " pred = model(x_batch)\n",
+ " # compute the lss\n",
+ " loss = loss_function(pred, y_batch)\n",
+ " # backward pass\n",
+ " loss.backward()\n",
+ " # SGD update\n",
+ " optimizer.step()\n",
+ "\n",
+ " # Run whole dataset to get statistics -- normally wouldn't do this\n",
+ " pred_train = model(x_train)\n",
+ " pred_val = model(x_val)\n",
+ " _, predicted_train_class = torch.max(pred_train.data, 1)\n",
+ " _, predicted_val_class = torch.max(pred_val.data, 1)\n",
+ " errors_train[epoch] = 100 - 100 * (predicted_train_class == y_train).float().sum() / len(y_train)\n",
+ " errors_val[epoch]= 100 - 100 * (predicted_val_class == y_val).float().sum() / len(y_val)\n",
+ " losses_train[epoch] = loss_function(pred_train, y_train).item()\n",
+ " losses_val[epoch]= loss_function(pred_val, y_val).item()\n",
+ " print(f'Epoch {epoch:5d}, train loss {losses_train[epoch]:.6f}, train error {errors_train[epoch]:3.2f}, val loss {losses_val[epoch]:.6f}, percent error {errors_val[epoch]:3.2f}')\n",
+ " \n",
+ " # tell scheduler to consider updating learning rate\n",
+ " scheduler.step()\n",
+ "\n",
+ "# Plot the results\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(errors_train,'r-',label='train')\n",
+ "ax.plot(errors_val,'b-',label='validation')\n",
+ "ax.set_ylim(0,100); ax.set_xlim(0,n_epoch)\n",
+ "ax.set_xlabel('Epoch'); ax.set_ylabel('Error')\n",
+ "ax.set_title('Part I: Validation Result %3.2f'%(errors_val[-1]))\n",
+ "ax.legend()\n",
+ "ax.plot([0,n_epoch],[37.45, 37.45],'k:') # Original results. You should be better than this!\n",
+ "plt.savefig('Coursework_I_Results.png',format='png')\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "NYw8I_3mmX5c",
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 1000
+ },
+ "outputId": "777de5cf-c31d-4519-c7d3-e363e08d394c"
+ },
+ "execution_count": 7,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Epoch 0, train loss 1.573925, train error 61.75, val loss 1.666198, percent error 67.85\n",
+ "Epoch 1, train loss 1.318168, train error 48.25, val loss 1.471983, percent error 57.95\n",
+ "Epoch 2, train loss 1.130921, train error 40.03, val loss 1.360329, percent error 53.45\n",
+ "Epoch 3, train loss 0.984176, train error 35.20, val loss 1.256625, percent error 48.10\n",
+ "Epoch 4, train loss 0.868384, train error 30.20, val loss 1.191934, percent error 45.30\n",
+ "Epoch 5, train loss 0.814746, train error 30.00, val loss 1.242606, percent error 48.25\n",
+ "Epoch 6, train loss 0.693850, train error 23.90, val loss 1.149572, percent error 43.70\n",
+ "Epoch 7, train loss 0.624264, train error 20.93, val loss 1.142540, percent error 41.20\n",
+ "Epoch 8, train loss 0.549854, train error 18.22, val loss 1.117410, percent error 40.55\n",
+ "Epoch 9, train loss 0.495227, train error 16.35, val loss 1.105867, percent error 40.25\n",
+ "Epoch 10, train loss 0.404633, train error 11.82, val loss 1.047640, percent error 37.90\n",
+ "Epoch 11, train loss 0.368226, train error 10.32, val loss 1.084517, percent error 40.00\n",
+ "Epoch 12, train loss 0.339168, train error 9.15, val loss 1.097698, percent error 38.50\n",
+ "Epoch 13, train loss 0.302940, train error 7.68, val loss 1.099108, percent error 37.25\n",
+ "Epoch 14, train loss 0.306518, train error 8.75, val loss 1.152268, percent error 39.40\n",
+ "Epoch 15, train loss 0.267522, train error 6.88, val loss 1.133403, percent error 38.20\n",
+ "Epoch 16, train loss 0.229632, train error 5.25, val loss 1.121083, percent error 37.15\n",
+ "Epoch 17, train loss 0.207498, train error 3.75, val loss 1.153062, percent error 38.00\n",
+ "Epoch 18, train loss 0.196556, train error 3.93, val loss 1.190135, percent error 37.40\n",
+ "Epoch 19, train loss 0.188664, train error 3.85, val loss 1.224324, percent error 37.10\n",
+ "Epoch 20, train loss 0.151122, train error 1.68, val loss 1.188515, percent error 36.50\n",
+ "Epoch 21, train loss 0.141133, train error 1.62, val loss 1.186356, percent error 36.30\n",
+ "Epoch 22, train loss 0.131978, train error 1.05, val loss 1.210334, percent error 37.10\n",
+ "Epoch 23, train loss 0.126643, train error 0.93, val loss 1.222403, percent error 37.15\n",
+ "Epoch 24, train loss 0.121445, train error 0.93, val loss 1.234944, percent error 36.60\n",
+ "Epoch 25, train loss 0.112892, train error 0.80, val loss 1.249163, percent error 36.35\n",
+ "Epoch 26, train loss 0.106721, train error 0.57, val loss 1.257951, percent error 37.40\n",
+ "Epoch 27, train loss 0.101724, train error 0.40, val loss 1.266331, percent error 36.90\n",
+ "Epoch 28, train loss 0.100189, train error 0.43, val loss 1.280694, percent error 37.50\n",
+ "Epoch 29, train loss 0.093124, train error 0.40, val loss 1.289725, percent error 37.50\n",
+ "Epoch 30, train loss 0.087898, train error 0.35, val loss 1.291468, percent error 36.75\n",
+ "Epoch 31, train loss 0.085375, train error 0.30, val loss 1.301522, percent error 37.60\n",
+ "Epoch 32, train loss 0.083599, train error 0.25, val loss 1.310020, percent error 37.40\n",
+ "Epoch 33, train loss 0.082141, train error 0.25, val loss 1.312388, percent error 37.00\n",
+ "Epoch 34, train loss 0.080171, train error 0.18, val loss 1.320177, percent error 37.05\n",
+ "Epoch 35, train loss 0.077832, train error 0.18, val loss 1.328110, percent error 37.40\n",
+ "Epoch 36, train loss 0.076884, train error 0.22, val loss 1.327245, percent error 36.75\n",
+ "Epoch 37, train loss 0.074366, train error 0.15, val loss 1.332270, percent error 37.35\n",
+ "Epoch 38, train loss 0.072928, train error 0.12, val loss 1.339683, percent error 37.25\n",
+ "Epoch 39, train loss 0.071071, train error 0.10, val loss 1.341762, percent error 37.20\n",
+ "Epoch 40, train loss 0.070039, train error 0.12, val loss 1.346855, percent error 37.35\n",
+ "Epoch 41, train loss 0.069533, train error 0.10, val loss 1.355226, percent error 37.45\n",
+ "Epoch 42, train loss 0.068655, train error 0.10, val loss 1.354576, percent error 37.60\n",
+ "Epoch 43, train loss 0.067851, train error 0.12, val loss 1.354539, percent error 37.05\n",
+ "Epoch 44, train loss 0.066937, train error 0.07, val loss 1.359383, percent error 37.70\n",
+ "Epoch 45, train loss 0.066291, train error 0.05, val loss 1.364250, percent error 37.55\n",
+ "Epoch 46, train loss 0.065502, train error 0.05, val loss 1.365553, percent error 37.55\n",
+ "Epoch 47, train loss 0.064782, train error 0.07, val loss 1.366146, percent error 37.55\n",
+ "Epoch 48, train loss 0.064185, train error 0.05, val loss 1.368595, percent error 37.35\n",
+ "Epoch 49, train loss 0.063420, train error 0.07, val loss 1.373254, percent error 37.45\n"
+ ]
+ },
+ {
+ "output_type": "display_data",
+ "data": {
+ "text/plain": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Coursework II -- Training hyperparameters\n",
+ "\n",
+ "The goal of the coursework is to modify a simple bit of numpy code that trains a network and measures the performance on a validation set for the MNist 1D dataset. \n",
+ "\n",
+ "In this coursework, you need to modify the **training hyperparameters** (only) to improve the performance over the current attempt. This could mean the training algorithm, learning rate, learning rate schedule, momentum term, initialization etc. \n",
+ "\n",
+ "You don't have to improve the performance much. A few tenths of a percent is fine. It just has to be better to get full marks.\n",
+ "\n",
+ "You will need to upload three things to Moodle:\n",
+ "1. The image that this notebook saves (click the folder icon on the left on colab to download it)\n",
+ "2. The lines of code you changed\n",
+ "3. The whole notebook as a .ipynb file. You can do this on the File menu\n",
+ "\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import os\n",
+ "import torch, torch.nn as nn\n",
+ "from torch.utils.data import TensorDataset, DataLoader\n",
+ "from torch.optim.lr_scheduler import StepLR\n",
+ "import matplotlib.pyplot as plt\n",
+ "import random"
+ ],
+ "metadata": {
+ "id": "YrXWAH7sUWvU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run this once to copy the train and validation data to your CoLab environment \n",
+ "# or download from my github to your local machine if you are doing this locally\n",
+ "if not os.path.exists('./train_data_x.npy'):\n",
+ " !wget https://github.com/udlbook/udlbook/raw/main/practicals/train_data_x.npy\n",
+ " !wget https://github.com/udlbook/udlbook/raw/main/practicals/train_data_y.npy\n",
+ " !wget https://github.com/udlbook/udlbook/raw/main/practicals/val_data_x.npy\n",
+ " !wget https://github.com/udlbook/udlbook/raw/main/practicals/val_data_y.npy "
+ ],
+ "metadata": {
+ "id": "wScBGXXFVadm"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Load in the data\n",
+ "train_data_x = np.load('train_data_x.npy')\n",
+ "train_data_y = np.load('train_data_y.npy')\n",
+ "val_data_x = np.load('val_data_x.npy')\n",
+ "val_data_y = np.load('val_data_y.npy')\n",
+ "# Print out sizes\n",
+ "print(\"Train data: %d examples (columns), each of which has %d dimensions (rows)\"%((train_data_x.shape[1],train_data_x.shape[0])))\n",
+ "print(\"Validation data: %d examples (columns), each of which has %d dimensions (rows)\"%((val_data_x.shape[1],val_data_x.shape[0])))"
+ ],
+ "metadata": {
+ "id": "8bKADvLHbiV5"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define the network"
+ ],
+ "metadata": {
+ "id": "_sFvRDGrl4qe"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# YOU SHOULD NOT CHANGE THIS CELL!\n",
+ "\n",
+ "# There are 40 input dimensions and 10 output dimensions for this data\n",
+ "# The inputs correspond to the 40 offsets in the MNIST1D template.\n",
+ "D_i = 40\n",
+ "# The outputs correspond to the 10 digits\n",
+ "D_o = 10\n",
+ "\n",
+ "# Number of hidden units in layers 1 and 2\n",
+ "D_1 = 100\n",
+ "D_2 = 100\n",
+ "\n",
+ "# create model with two hidden layers\n",
+ "model = nn.Sequential(\n",
+ "nn.Linear(D_i, D_1),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(D_1, D_2),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(D_2, D_o))"
+ ],
+ "metadata": {
+ "id": "FslroPJJffrh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# He initialization of weights\n",
+ "def weights_init(layer_in):\n",
+ " if isinstance(layer_in, nn.Linear):\n",
+ " nn.init.kaiming_uniform_(layer_in.weight)\n",
+ " layer_in.bias.data.fill_(0.0)"
+ ],
+ "metadata": {
+ "id": "YgLaex1pfhqz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# You need all this stuff to ensure that PyTorch is deterministic\n",
+ "def set_seed(seed):\n",
+ " torch.manual_seed(seed)\n",
+ " torch.cuda.manual_seed_all(seed)\n",
+ " torch.backends.cudnn.deterministic = True\n",
+ " torch.backends.cudnn.benchmark = False\n",
+ " np.random.seed(seed)\n",
+ " random.seed(seed)\n",
+ " os.environ['PYTHONHASHSEED'] = str(seed)"
+ ],
+ "metadata": {
+ "id": "zXRmxCQNnL_M"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set seed so always get same result (do not change)\n",
+ "set_seed(1)\n",
+ "\n",
+ "# choose cross entropy loss function (equation 5.24 in the loss notes)\n",
+ "loss_function = nn.CrossEntropyLoss()\n",
+ "# construct SGD optimizer and initialize learning rate and momentum\n",
+ "optimizer = torch.optim.SGD(model.parameters(), lr = 0.05, momentum=0.9)\n",
+ "# object that decreases learning rate by half every 10 epochs\n",
+ "scheduler = StepLR(optimizer, step_size=10, gamma=0.5)\n",
+ "# create 100 dummy data points and store in data loader class\n",
+ "x_train = torch.tensor(train_data_x.transpose().astype('float32'))\n",
+ "y_train = torch.tensor(train_data_y.astype('long'))\n",
+ "x_val= torch.tensor(val_data_x.transpose().astype('float32'))\n",
+ "y_val = torch.tensor(val_data_y.astype('long'))\n",
+ "\n",
+ "# load the data into a class that creates the batches\n",
+ "data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
+ "\n",
+ "# Initialize model weights\n",
+ "model.apply(weights_init)\n",
+ "\n",
+ "# loop over the dataset n_epoch times\n",
+ "n_epoch = 50\n",
+ "# store the loss and the % correct at each epoch\n",
+ "losses_train = np.zeros((n_epoch))\n",
+ "errors_train = np.zeros((n_epoch))\n",
+ "losses_val = np.zeros((n_epoch))\n",
+ "errors_val = np.zeros((n_epoch))\n",
+ "\n",
+ "for epoch in range(n_epoch):\n",
+ " # loop over batches\n",
+ " for i, data in enumerate(data_loader):\n",
+ " # retrieve inputs and labels for this batch\n",
+ " x_batch, y_batch = data\n",
+ " # zero the parameter gradients\n",
+ " optimizer.zero_grad()\n",
+ " # forward pass -- calculate model output\n",
+ " pred = model(x_batch)\n",
+ " # compute the lss\n",
+ " loss = loss_function(pred, y_batch)\n",
+ " # backward pass\n",
+ " loss.backward()\n",
+ " # SGD update\n",
+ " optimizer.step()\n",
+ "\n",
+ " # Run whole dataset to get statistics -- normally wouldn't do this\n",
+ " pred_train = model(x_train)\n",
+ " pred_val = model(x_val)\n",
+ " _, predicted_train_class = torch.max(pred_train.data, 1)\n",
+ " _, predicted_val_class = torch.max(pred_val.data, 1)\n",
+ " errors_train[epoch] = 100 - 100 * (predicted_train_class == y_train).float().sum() / len(y_train)\n",
+ " errors_val[epoch]= 100 - 100 * (predicted_val_class == y_val).float().sum() / len(y_val)\n",
+ " losses_train[epoch] = loss_function(pred_train, y_train).item()\n",
+ " losses_val[epoch]= loss_function(pred_val, y_val).item()\n",
+ " print(f'Epoch {epoch:5d}, train loss {losses_train[epoch]:.6f}, train error {errors_train[epoch]:3.2f}, val loss {losses_val[epoch]:.6f}, percent error {errors_val[epoch]:3.2f}')\n",
+ " \n",
+ " # tell scheduler to consider updating learning rate\n",
+ " scheduler.step()\n",
+ "\n",
+ "# Plot the results\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(errors_train,'r-',label='train')\n",
+ "ax.plot(errors_val,'b-',label='validation')\n",
+ "ax.set_ylim(0,100); ax.set_xlim(0,n_epoch)\n",
+ "ax.set_xlabel('Epoch'); ax.set_ylabel('Error')\n",
+ "ax.set_title('Part II: Validation Result %3.2f'%(errors_val[-1]))\n",
+ "ax.legend()\n",
+ "ax.plot([0,n_epoch],[37.45, 37.45],'k:') # Original results. You should be better than this!\n",
+ "plt.savefig('Coursework_II_Results.png',format='png')\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "NYw8I_3mmX5c"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Leave this all commented for now\n",
+ "# We'll see how well you did on the test data after the coursework is submitted\n",
+ "\n",
+ "# if not os.path.exists('./test_data_x.npy'):\n",
+ "# !wget https://github.com/udlbook/udlbook/raw/main/practicals/test_data_x.npy\n",
+ "# !wget https://github.com/udlbook/udlbook/raw/main/practicals/test_data_y.npy\n",
+ "\n",
+ "\n",
+ "# # I haven't given you this yet, leave commented\n",
+ "# test_data_x = np.load('test_data_x.npy')\n",
+ "# test_data_y = np.load('test_data_y.npy')\n",
+ "# x_test = torch.tensor(test_data_x.transpose().astype('float32'))\n",
+ "# y_test = torch.tensor(test_data_y.astype('long'))\n",
+ "# pred_test = model(x_test)\n",
+ "# _, predicted_test_class = torch.max(pred_test.data, 1)\n",
+ "# errors_test = 100 - 100 * (predicted_test_class == y_test).float().sum() / len(y_test)\n",
+ "# print(\"Test error = %3.3f\"%(errors_test))"
+ ],
+ "metadata": {
+ "id": "O7nBz-R84QdJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "zXccksoD1Eww"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
diff --git a/CM20315/CM20315_Coursework_III.ipynb b/CM20315/CM20315_Coursework_III.ipynb
new file mode 100644
index 0000000..639d64a
--- /dev/null
+++ b/CM20315/CM20315_Coursework_III.ipynb
@@ -0,0 +1,290 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyMcvUAtcMw0yuDfUkbsEDLD",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Coursework III -- Regularization\n",
+ "\n",
+ "The goal of the coursework is to modify a simple bit of numpy code that trains a network and measures the performance on a validation set for the MNist 1D dataset. \n",
+ "\n",
+ "In this coursework, you need add **regularization** of some kind to improve the performance. Anything from chapter 8 of the book or anything else you can find is fine *except* early stopping. You must not change the model hyperparameters or the training algorithm.\n",
+ "\n",
+ "You don't have to improve the performance much. A few tenths of a percent is fine. It just has to be better to get full marks.\n",
+ "\n",
+ "You will need to upload three things to Moodle:\n",
+ "1. The image that this notebook saves (click the folder icon on the left on colab to download it)\n",
+ "2. The lines of code you changed\n",
+ "3. The whole notebook as a .ipynb file. You can do this on the File menu\n",
+ "\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import os\n",
+ "import torch, torch.nn as nn\n",
+ "from torch.utils.data import TensorDataset, DataLoader\n",
+ "from torch.optim.lr_scheduler import StepLR\n",
+ "import matplotlib.pyplot as plt\n",
+ "import random"
+ ],
+ "metadata": {
+ "id": "YrXWAH7sUWvU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run this once to copy the train and validation data to your CoLab environment \n",
+ "# or download from my github to your local machine if you are doing this locally\n",
+ "if not os.path.exists('./train_data_x.npy'):\n",
+ " !wget https://github.com/udlbook/udlbook/raw/main/practicals/train_data_x.npy\n",
+ " !wget https://github.com/udlbook/udlbook/raw/main/practicals/train_data_y.npy\n",
+ " !wget https://github.com/udlbook/udlbook/raw/main/practicals/val_data_x.npy\n",
+ " !wget https://github.com/udlbook/udlbook/raw/main/practicals/val_data_y.npy "
+ ],
+ "metadata": {
+ "id": "wScBGXXFVadm"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Load in the data\n",
+ "train_data_x = np.load('train_data_x.npy')\n",
+ "train_data_y = np.load('train_data_y.npy')\n",
+ "val_data_x = np.load('val_data_x.npy')\n",
+ "val_data_y = np.load('val_data_y.npy')\n",
+ "# Print out sizes\n",
+ "print(\"Train data: %d examples (columns), each of which has %d dimensions (rows)\"%((train_data_x.shape[1],train_data_x.shape[0])))\n",
+ "print(\"Validation data: %d examples (columns), each of which has %d dimensions (rows)\"%((val_data_x.shape[1],val_data_x.shape[0])))"
+ ],
+ "metadata": {
+ "id": "8bKADvLHbiV5"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define the network"
+ ],
+ "metadata": {
+ "id": "_sFvRDGrl4qe"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# There are 40 input dimensions and 10 output dimensions for this data\n",
+ "# The inputs correspond to the 40 offsets in the MNIST1D template.\n",
+ "D_i = 40\n",
+ "# The outputs correspond to the 10 digits\n",
+ "D_o = 10\n",
+ "\n",
+ "# Number of hidden units in layers 1 and 2\n",
+ "D_1 = 100\n",
+ "D_2 = 100\n",
+ "\n",
+ "# create model with two hidden layers\n",
+ "model = nn.Sequential(\n",
+ "nn.Linear(D_i, D_1),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(D_1, D_2),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(D_2, D_o))"
+ ],
+ "metadata": {
+ "id": "FslroPJJffrh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# He initialization of weights\n",
+ "def weights_init(layer_in):\n",
+ " if isinstance(layer_in, nn.Linear):\n",
+ " nn.init.kaiming_uniform_(layer_in.weight)\n",
+ " layer_in.bias.data.fill_(0.0)"
+ ],
+ "metadata": {
+ "id": "YgLaex1pfhqz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# You need all this stuff to ensure that PyTorch is deterministic\n",
+ "def set_seed(seed):\n",
+ " torch.manual_seed(seed)\n",
+ " torch.cuda.manual_seed_all(seed)\n",
+ " torch.backends.cudnn.deterministic = True\n",
+ " torch.backends.cudnn.benchmark = False\n",
+ " np.random.seed(seed)\n",
+ " random.seed(seed)\n",
+ " os.environ['PYTHONHASHSEED'] = str(seed)"
+ ],
+ "metadata": {
+ "id": "zXRmxCQNnL_M"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set seed so always get same result (do not change)\n",
+ "set_seed(1)\n",
+ "\n",
+ "# choose cross entropy loss function (equation 5.24 in the loss notes)\n",
+ "loss_function = nn.CrossEntropyLoss()\n",
+ "# construct SGD optimizer and initialize learning rate and momentum\n",
+ "optimizer = torch.optim.SGD(model.parameters(), lr = 0.05, momentum=0.9)\n",
+ "# object that decreases learning rate by half every 10 epochs\n",
+ "scheduler = StepLR(optimizer, step_size=10, gamma=0.5)\n",
+ "# create 100 dummy data points and store in data loader class\n",
+ "x_train = torch.tensor(train_data_x.transpose().astype('float32'))\n",
+ "y_train = torch.tensor(train_data_y.astype('long'))\n",
+ "x_val= torch.tensor(val_data_x.transpose().astype('float32'))\n",
+ "y_val = torch.tensor(val_data_y.astype('long'))\n",
+ "\n",
+ "# load the data into a class that creates the batches\n",
+ "data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
+ "\n",
+ "# Initialize model weights\n",
+ "model.apply(weights_init)\n",
+ "\n",
+ "# loop over the dataset n_epoch times\n",
+ "n_epoch = 50\n",
+ "# store the loss and the % correct at each epoch\n",
+ "losses_train = np.zeros((n_epoch))\n",
+ "errors_train = np.zeros((n_epoch))\n",
+ "losses_val = np.zeros((n_epoch))\n",
+ "errors_val = np.zeros((n_epoch))\n",
+ "\n",
+ "for epoch in range(n_epoch):\n",
+ " # loop over batches\n",
+ " for i, data in enumerate(data_loader):\n",
+ " # retrieve inputs and labels for this batch\n",
+ " x_batch, y_batch = data\n",
+ " # zero the parameter gradients\n",
+ " optimizer.zero_grad()\n",
+ " # forward pass -- calculate model output\n",
+ " pred = model(x_batch)\n",
+ " # compute the lss\n",
+ " loss = loss_function(pred, y_batch)\n",
+ " # backward pass\n",
+ " loss.backward()\n",
+ " # SGD update\n",
+ " optimizer.step()\n",
+ "\n",
+ " # Run whole dataset to get statistics -- normally wouldn't do this\n",
+ " pred_train = model(x_train)\n",
+ " pred_val = model(x_val)\n",
+ " _, predicted_train_class = torch.max(pred_train.data, 1)\n",
+ " _, predicted_val_class = torch.max(pred_val.data, 1)\n",
+ " errors_train[epoch] = 100 - 100 * (predicted_train_class == y_train).float().sum() / len(y_train)\n",
+ " errors_val[epoch]= 100 - 100 * (predicted_val_class == y_val).float().sum() / len(y_val)\n",
+ " losses_train[epoch] = loss_function(pred_train, y_train).item()\n",
+ " losses_val[epoch]= loss_function(pred_val, y_val).item()\n",
+ " print(f'Epoch {epoch:5d}, train loss {losses_train[epoch]:.6f}, train error {errors_train[epoch]:3.2f}, val loss {losses_val[epoch]:.6f}, percent error {errors_val[epoch]:3.2f}')\n",
+ " \n",
+ " # tell scheduler to consider updating learning rate\n",
+ " scheduler.step()\n",
+ "\n",
+ "# Plot the results\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(errors_train,'r-',label='train')\n",
+ "ax.plot(errors_val,'b-',label='validation')\n",
+ "ax.set_ylim(0,100); ax.set_xlim(0,n_epoch)\n",
+ "ax.set_xlabel('Epoch'); ax.set_ylabel('Error')\n",
+ "ax.set_title('Part III: Validation Result %3.2f'%(errors_val[-1]))\n",
+ "ax.legend()\n",
+ "ax.plot([0,n_epoch],[37.45, 37.45],'k:') # Original results. You should be better than this!\n",
+ "plt.savefig('Coursework_III_Results.png',format='png')\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "NYw8I_3mmX5c"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Leave this all commented for now\n",
+ "# We'll see how well you did on the test data after the coursework is submitted\n",
+ "\n",
+ "# if not os.path.exists('./test_data_x.npy'):\n",
+ "# !wget https://github.com/udlbook/udlbook/raw/main/practicals/test_data_x.npy\n",
+ "# !wget https://github.com/udlbook/udlbook/raw/main/practicals/test_data_y.npy\n",
+ "\n",
+ "\n",
+ "# # I haven't given you this yet, leave commented\n",
+ "# test_data_x = np.load('test_data_x.npy')\n",
+ "# test_data_y = np.load('test_data_y.npy')\n",
+ "# x_test = torch.tensor(test_data_x.transpose().astype('float32'))\n",
+ "# y_test = torch.tensor(test_data_y.astype('long'))\n",
+ "# pred_test = model(x_test)\n",
+ "# _, predicted_test_class = torch.max(pred_test.data, 1)\n",
+ "# errors_test = 100 - 100 * (predicted_test_class == y_test).float().sum() / len(y_test)\n",
+ "# print(\"Test error = %3.3f\"%(errors_test))"
+ ],
+ "metadata": {
+ "id": "O7nBz-R84QdJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "zXccksoD1Eww"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
diff --git a/CM20315/CM20315_Coursework_IV.ipynb b/CM20315/CM20315_Coursework_IV.ipynb
new file mode 100644
index 0000000..dfdb5e9
--- /dev/null
+++ b/CM20315/CM20315_Coursework_IV.ipynb
@@ -0,0 +1,222 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyNb1nfymw3lpvyBHaCFRvMI",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Coursework IV\n",
+ "\n",
+ "This coursework explores the geometry of high dimensional spaces. It doesn't behave how you would expect and all your intuitions are wrong! You will write code and it will give you three numerical answers that you need to type into Moodle."
+ ],
+ "metadata": {
+ "id": "EjLK-kA1KnYX"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4ESMmnkYEVAb"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "import scipy.special as sci"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Part (a)\n",
+ "\n",
+ "In part (a) of the practical, we investigate how close random points are in 2D, 100D, and 1000D. In each case, we generate 1000 points and calculate the Euclidean distance between each pair. You should find that in 1000D, the furthest two points are only slightly further apart than the nearest points. Weird!"
+ ],
+ "metadata": {
+ "id": "MonbPEitLNgN"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Fix the random seed so we all have the same random numbers\n",
+ "np.random.seed(0)\n",
+ "n_data = 1000\n",
+ "# Create 1000 data examples (columns) each with 2 dimensions (rows)\n",
+ "n_dim = 2\n",
+ "x_2D = np.random.normal(size=(n_dim,n_data))\n",
+ "# Create 1000 data examples (columns) each with 100 dimensions (rows)\n",
+ "n_dim = 100\n",
+ "x_100D = np.random.normal(size=(n_dim,n_data))\n",
+ "# Create 1000 data examples (columns) each with 1000 dimensions (rows)\n",
+ "n_dim = 1000\n",
+ "x_1000D = np.random.normal(size=(n_dim,n_data))\n",
+ "\n",
+ "# These values should be the same, otherwise your answer will be wrong\n",
+ "# Get in touch if they are not!\n",
+ "print('Sum of your data is %3.3f, Should be %3.3f'%(np.sum(x_1000D),1036.321))"
+ ],
+ "metadata": {
+ "id": "vZSHVmcWEk14"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def distance_ratio(x):\n",
+ " # TODO -- replace the two lines below to calculate the largest and smallest Euclidean distance between \n",
+ " # the data points in the columns of x. DO NOT include the distance between the data point\n",
+ " # and itself (which is obviously zero)\n",
+ " smallest_dist = 1.0\n",
+ " largest_dist = 1.0\n",
+ " \n",
+ " # Calculate the ratio and return\n",
+ " dist_ratio = largest_dist / smallest_dist\n",
+ " return dist_ratio"
+ ],
+ "metadata": {
+ "id": "PhVmnUs8ErD9"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "print('Ratio of largest to smallest distance 2D: %3.3f'%(distance_ratio(x_2D)))\n",
+ "print('Ratio of largest to smallest distance 100D: %3.3f'%(distance_ratio(x_100D)))\n",
+ "print('Ratio of largest to smallest distance 1000D: %3.3f'%(distance_ratio(x_1000D)))\n",
+ "print('**Note down the last of these three numbers, you will need to submit it for your coursework**')"
+ ],
+ "metadata": {
+ "id": "0NdPxfn5GQuJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Part (b)\n",
+ "\n",
+ "In part (b) of the practical we calculate the volume of a hypersphere of radius 0.5 (i.e., of diameter 1) as a function of the radius. You will find that the volume decreases to almost nothing in high dimensions. All of the volume is in the corners of the unit hypercube (which always has volume 1). Double weird.\n",
+ "\n",
+ "Note that you you can check your answer by doing the calculation for 2D using the standard formula for the area of a circle and making sure it matches."
+ ],
+ "metadata": {
+ "id": "b2FYKV1SL4Z7"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def volume_of_hypersphere(diameter, dimensions):\n",
+ " # Formula iven in Problem 8.7 of the notes in Moodle (probably a different problem number on book site)\n",
+ " # You will need sci.special.gamma()\n",
+ " # Check out: https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.gamma.html\n",
+ " # Also use this value for pi\n",
+ " pi = np.pi\n",
+ " # TODO replace this code with formula for the volume of a hypersphere\n",
+ " volume = 1.0\n",
+ "\n",
+ " return volume\n",
+ " "
+ ],
+ "metadata": {
+ "id": "CZoNhD8XJaHR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "diameter = 1.0\n",
+ "for c_dim in range(1,11):\n",
+ " print(\"Volume of unit diameter hypersphere in %d dimensions is %3.3f\"%(c_dim, volume_of_hypersphere(diameter, c_dim)))\n",
+ "print('**Note down the last of these ten numbers, you will need to submit it for your coursework**')"
+ ],
+ "metadata": {
+ "id": "fNTBlg_GPEUh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Part (c)\n",
+ "\n",
+ "In part (c) of the coursework, you will calculate what proportion of the volume of a hypersphere is in the outer 1% of the radius/diameter. Calculate the volume of a hypersphere and then the volume of a hypersphere with 0.99 of the radius and then figure out the proportion (a number between 0 and 1). You'll see that by the time we get to 300 dimensions most of the volume is in the outer 1 percent. Extremely weird!"
+ ],
+ "metadata": {
+ "id": "GdyMeOBmoXyF"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def get_prop_of_volume_in_outer_1_percent(dimension):\n",
+ " # TODO -- replace this line\n",
+ " proportion = 1.0\n",
+ "\n",
+ " return proportion"
+ ],
+ "metadata": {
+ "id": "8_CxZ2AIpQ8w"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# While we're here, let's look at how much of the volume is in the outer 1% of the radius\n",
+ "for c_dim in [1,2,10,20,50,100,150,200,250,300]:\n",
+ " print('Proportion of volume in outer 1 percent of radius in %d dimensions =%3.3f'%(c_dim, get_prop_of_volume_in_outer_1_percent(c_dim)))\n",
+ "print('**Note down the last of these ten numbers, you will need to submit it for your coursework**')"
+ ],
+ "metadata": {
+ "id": "LtMDIn2qPVfJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "Qbxb-eHHQCS7"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
diff --git a/CM20315/CM20315_Deep.ipynb b/CM20315/CM20315_Deep.ipynb
new file mode 100644
index 0000000..1607e2c
--- /dev/null
+++ b/CM20315/CM20315_Deep.ipynb
@@ -0,0 +1,448 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyOtP/O21RVLxAeEBIwV0aZt",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Deep neural networks**\n",
+ "\n",
+ "In this notebook, we'll experiment with feeding one neural network into another as in figure 4.1 from the book."
+ ],
+ "metadata": {
+ "id": "MaKn8CFlzN8E"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "8ClURpZQzI6L"
+ },
+ "outputs": [],
+ "source": [
+ "# Imports math library\n",
+ "import numpy as np\n",
+ "# Imports plotting library\n",
+ "import matplotlib.pyplot as plt"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the Rectified Linear Unit (ReLU) function\n",
+ "def ReLU(preactivation):\n",
+ " activation = preactivation.clip(0.0)\n",
+ " return activation"
+ ],
+ "metadata": {
+ "id": "YdmveeAUz4YG"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define a shallow neural network with, one input, one output, and three hidden units\n",
+ "def shallow_1_1_3(x, activation_fn, phi_0,phi_1,phi_2,phi_3, theta_10, theta_11, theta_20, theta_21, theta_30, theta_31):\n",
+ " # Initial lines\n",
+ " pre_1 = theta_10 + theta_11 * x\n",
+ " pre_2 = theta_20 + theta_21 * x\n",
+ " pre_3 = theta_30 + theta_31 * x\n",
+ " # Activation functions\n",
+ " act_1 = activation_fn(pre_1)\n",
+ " act_2 = activation_fn(pre_2)\n",
+ " act_3 = activation_fn(pre_3)\n",
+ " # Weight activations\n",
+ " w_act_1 = phi_1 * act_1\n",
+ " w_act_2 = phi_2 * act_2\n",
+ " w_act_3 = phi_3 * act_3\n",
+ " # Combine weighted activation and add y offset\n",
+ " y = phi_0 + w_act_1 + w_act_2 + w_act_3\n",
+ " # Return everything we have calculated\n",
+ " return y, pre_1, pre_2, pre_3, act_1, act_2, act_3, w_act_1, w_act_2, w_act_3"
+ ],
+ "metadata": {
+ "id": "ximCLwIfz8kj"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# # Plot the shallow neural network. We'll assume input in is range [-1,1] and output [-1,1]\n",
+ "# If the plot_all flag is set to true, then we'll plot all the intermediate stages as in Figure 3.3 \n",
+ "def plot_neural(x, y, pre_1, pre_2, pre_3, act_1, act_2, act_3, w_act_1, w_act_2, w_act_3, plot_all=False, x_data=None, y_data=None):\n",
+ "\n",
+ " # Plot intermediate plots if flag set\n",
+ " if plot_all:\n",
+ " fig, ax = plt.subplots(3,3)\n",
+ " fig.set_size_inches(8.5, 8.5)\n",
+ " fig.tight_layout(pad=3.0)\n",
+ " ax[0,0].plot(x,pre_1,'r-'); ax[0,0].set_ylabel('Preactivation')\n",
+ " ax[0,1].plot(x,pre_2,'b-'); ax[0,1].set_ylabel('Preactivation')\n",
+ " ax[0,2].plot(x,pre_3,'g-'); ax[0,2].set_ylabel('Preactivation')\n",
+ " ax[1,0].plot(x,act_1,'r-'); ax[1,0].set_ylabel('Activation')\n",
+ " ax[1,1].plot(x,act_2,'b-'); ax[1,1].set_ylabel('Activation')\n",
+ " ax[1,2].plot(x,act_3,'g-'); ax[1,2].set_ylabel('Activation')\n",
+ " ax[2,0].plot(x,w_act_1,'r-'); ax[2,0].set_ylabel('Weighted Act')\n",
+ " ax[2,1].plot(x,w_act_2,'b-'); ax[2,1].set_ylabel('Weighted Act')\n",
+ " ax[2,2].plot(x,w_act_3,'g-'); ax[2,2].set_ylabel('Weighted Act')\n",
+ "\n",
+ " for plot_y in range(3):\n",
+ " for plot_x in range(3):\n",
+ " ax[plot_y,plot_x].set_xlim([-1,1]);ax[plot_x,plot_y].set_ylim([-1,1])\n",
+ " ax[plot_y,plot_x].set_aspect(1.0)\n",
+ " ax[2,plot_y].set_xlabel('Input, $x$');\n",
+ " plt.show()\n",
+ "\n",
+ " fig, ax = plt.subplots()\n",
+ " ax.plot(x,y)\n",
+ " ax.set_xlabel('Input'); ax.set_ylabel('Output')\n",
+ " ax.set_xlim([-1,1]);ax.set_ylim([-1,1])\n",
+ " ax.set_aspect(1.0)\n",
+ " if x_data is not None:\n",
+ " ax.plot(x_data, y_data, 'mo')\n",
+ " for i in range(len(x_data)):\n",
+ " ax.plot(x_data[i], y_data[i],)\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "btrt7BX20gKD"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's define two networks. We'll put the prefixes n1_ and n2_ before all the variables to make it clear which network is which. We'll just consider the inputs and outputs over the range [-1,1]. If you set the \"plot_all\" flat to True, you can see the details of how they were created."
+ ],
+ "metadata": {
+ "id": "LxBJCObC-NTY"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now lets define some parameters and run the first neural network\n",
+ "n1_theta_10 = 0.0 ; n1_theta_11 = -1.0\n",
+ "n1_theta_20 = 0 ; n1_theta_21 = 1.0\n",
+ "n1_theta_30 = -0.67 ; n1_theta_31 = 1.0\n",
+ "n1_phi_0 = 1.0; n1_phi_1 = -2.0; n1_phi_2 = -3.0; n1_phi_3 = 9.3\n",
+ "\n",
+ "# Define a range of input values\n",
+ "n1_in = np.arange(-1,1,0.01)\n",
+ "\n",
+ "# We run the neural network for each of these input values\n",
+ "n1_out, pre_1, pre_2, pre_3, act_1, act_2, act_3, w_act_1, w_act_2, w_act_3 = \\\n",
+ " shallow_1_1_3(n1_in, ReLU, n1_phi_0, n1_phi_1, n1_phi_2, n1_phi_3, n1_theta_10, n1_theta_11, n1_theta_20, n1_theta_21, n1_theta_30, n1_theta_31)\n",
+ "# And then plot it\n",
+ "plot_neural(n1_in, n1_out, pre_1, pre_2, pre_3, act_1, act_2, act_3, w_act_1, w_act_2, w_act_3, plot_all=False)"
+ ],
+ "metadata": {
+ "id": "JRebvurv22pT"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now lets define some parameters and run the second neural network\n",
+ "n2_theta_10 = -0.6 ; n2_theta_11 = -1.0\n",
+ "n2_theta_20 = 0.2 ; n2_theta_21 = 1.0\n",
+ "n2_theta_30 = -0.5 ; n2_theta_31 = 1.0\n",
+ "n2_phi_0 = 0.5; n2_phi_1 = -1.0; n2_phi_2 = -1.5; n2_phi_3 = 2.0\n",
+ "\n",
+ "# Define a range of input values\n",
+ "n2_in = np.arange(-1,1,0.01)\n",
+ "\n",
+ "# We run the neural network for each of these input values\n",
+ "n2_out, pre_1, pre_2, pre_3, act_1, act_2, act_3, w_act_1, w_act_2, w_act_3 = \\\n",
+ " shallow_1_1_3(n2_in, ReLU, n2_phi_0, n2_phi_1, n2_phi_2, n2_phi_3, n2_theta_10, n2_theta_11, n2_theta_20, n2_theta_21, n2_theta_30, n2_theta_31)\n",
+ "# And then plot it\n",
+ "plot_neural(n2_in, n2_out, pre_1, pre_2, pre_3, act_1, act_2, act_3, w_act_1, w_act_2, w_act_3, plot_all=False)"
+ ],
+ "metadata": {
+ "id": "ZRjWu8i9239X"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now we'll consider feeding output of the first network into the second one."
+ ],
+ "metadata": {
+ "id": "qOcj2Rof-o20"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# # Plot two shallow neural networks and the composition of the two \n",
+ "def plot_neural_two_components(x_in, net1_out, net2_out, net12_out=None):\n",
+ "\n",
+ " # Plot the two networks separately\n",
+ " fig, ax = plt.subplots(1,2)\n",
+ " fig.set_size_inches(8.5, 8.5)\n",
+ " fig.tight_layout(pad=3.0)\n",
+ " ax[0].plot(x_in, net1_out,'r-')\n",
+ " ax[0].set_xlabel('Net 1 input'); ax[0].set_ylabel('Net 1 output')\n",
+ " ax[0].set_xlim([-1,1]);ax[0].set_ylim([-1,1])\n",
+ " ax[0].set_aspect(1.0)\n",
+ " ax[1].plot(x_in, net2_out,'b-')\n",
+ " ax[1].set_xlabel('Net 2 input'); ax[1].set_ylabel('Net 2 output')\n",
+ " ax[1].set_xlim([-1,1]);ax[1].set_ylim([-1,1])\n",
+ " ax[1].set_aspect(1.0)\n",
+ " plt.show()\n",
+ "\n",
+ " if net12_out is not None:\n",
+ " # Plot their composition\n",
+ " fig, ax = plt.subplots()\n",
+ " ax.plot(x_in ,net12_out,'g-')\n",
+ " ax.set_xlabel('Net 1 Input'); ax.set_ylabel('Net 2 Output')\n",
+ " ax.set_xlim([-1,1]);ax.set_ylim([-1,1])\n",
+ " ax.set_aspect(1.0)\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "ZB2HTalOE40X"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Display the two inputs\n",
+ "x = np.arange(-1,1,0.001)\n",
+ "# We run the first and second neural networks for each of these input values\n",
+ "net1_out, *_ = shallow_1_1_3(x, ReLU, n1_phi_0, n1_phi_1, n1_phi_2, n1_phi_3, n1_theta_10, n1_theta_11, n1_theta_20, n1_theta_21, n1_theta_30, n1_theta_31)\n",
+ "net2_out, *_ = shallow_1_1_3(x, ReLU, n2_phi_0, n2_phi_1, n2_phi_2, n2_phi_3, n2_theta_10, n2_theta_11, n2_theta_20, n2_theta_21, n2_theta_30, n2_theta_31)\n",
+ "# Plot both graphs\n",
+ "plot_neural_two_components(x, net1_out, net2_out)"
+ ],
+ "metadata": {
+ "id": "K6Tmecgu7uqt"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO \n",
+ "# Take a piece of paper and draw what you think will happen when we feed the \n",
+ "# output of the first network into the second one. Draw the relationship between\n",
+ "# the input of the first network and the output of the second one."
+ ],
+ "metadata": {
+ "id": "NUQVop9-Xta1"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now let's see if your predictions were right \n",
+ "\n",
+ "# TODO feed the output of first network into second network (replace this line)\n",
+ "net12_out = np.zeros_like(x)\n",
+ "# Plot all three graphs\n",
+ "plot_neural_two_components(x, net1_out, net2_out, net12_out)"
+ ],
+ "metadata": {
+ "id": "Yq7GH-MCIyPI"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now we'll change things a up a bit. What happens if we change the second network? (note the *-1 change)\n",
+ "net1_out, *_ = shallow_1_1_3(x, ReLU, n1_phi_0, n1_phi_1, n1_phi_2, n1_phi_3, n1_theta_10, n1_theta_11, n1_theta_20, n1_theta_21, n1_theta_30, n1_theta_31)\n",
+ "net2_out, *_ = shallow_1_1_3(x, ReLU, n2_phi_0, n2_phi_1*-1, n2_phi_2, n2_phi_3, n2_theta_10, n2_theta_11, n2_theta_20, n2_theta_21, n2_theta_30, n2_theta_31)\n",
+ "plot_neural_two_components(x, net1_out, net2_out)"
+ ],
+ "metadata": {
+ "id": "BMlLkLbdEuPu"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO \n",
+ "# Take a piece of paper and draw what you think will happen when we feed the \n",
+ "# output of the first network into the second one now that we have changed it. Draw the relationship between\n",
+ "# the input of the first network and the output of the second one."
+ ],
+ "metadata": {
+ "id": "Of6jVXLTJ688"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# When you have a prediction, run this code to see if you were right\n",
+ "net12_out, *_ = shallow_1_1_3(net1_out, ReLU, n2_phi_0, n2_phi_1*-1, n2_phi_2, n2_phi_3, n2_theta_10, n2_theta_11, n2_theta_20, n2_theta_21, n2_theta_30, n2_theta_31)\n",
+ "plot_neural_two_components(x, net1_out, net2_out, net12_out)"
+ ],
+ "metadata": {
+ "id": "PbbSCaSeK6SM"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's change things again. What happens if we change the firsrt network? (note the changes)\n",
+ "net1_out, *_ = shallow_1_1_3(x, ReLU, n1_phi_0, n1_phi_1*0.5, n1_phi_2, n1_phi_3, n1_theta_10, n1_theta_11, n1_theta_20, n1_theta_21, n1_theta_30, n1_theta_31)\n",
+ "net2_out, *_ = shallow_1_1_3(x, ReLU, n2_phi_0, n2_phi_1, n2_phi_2, n2_phi_3, n2_theta_10, n2_theta_11, n2_theta_20, n2_theta_21, n2_theta_30, n2_theta_31)\n",
+ "plot_neural_two_components(x, net1_out, net2_out)"
+ ],
+ "metadata": {
+ "id": "b39mcSGFK9Fd"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO \n",
+ "# Take a piece of paper and draw what you think will happen when we feed the \n",
+ "# output of the first network now we have changed it into the original second network. Draw the relationship between\n",
+ "# the input of the first network and the output of the second one."
+ ],
+ "metadata": {
+ "id": "MhO40cC_LW9I"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# When you have a prediction, run this code to see if you were right\n",
+ "net12_out, *_ = shallow_1_1_3(net1_out, ReLU, n2_phi_0, n2_phi_1, n2_phi_2, n2_phi_3, n2_theta_10, n2_theta_11, n2_theta_20, n2_theta_21, n2_theta_30, n2_theta_31)\n",
+ "plot_neural_two_components(x, net1_out, net2_out, net12_out)"
+ ],
+ "metadata": {
+ "id": "Akwo-hnPLkNr"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's change things again. What happens if the first network and second networks are the same?\n",
+ "net1_out, *_ = shallow_1_1_3(x, ReLU, n1_phi_0, n1_phi_1, n1_phi_2, n1_phi_3, n1_theta_10, n1_theta_11, n1_theta_20, n1_theta_21, n1_theta_30, n1_theta_31)\n",
+ "net2_out_new, *_ = shallow_1_1_3(x, ReLU, n1_phi_0, n1_phi_1, n1_phi_2, n1_phi_3, n1_theta_10, n1_theta_11, n1_theta_20, n1_theta_21, n1_theta_30, n1_theta_31)\n",
+ "plot_neural_two_components(x, net1_out, net2_out_new)"
+ ],
+ "metadata": {
+ "id": "TJ7wXKpRLl_E"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO \n",
+ "# Take a piece of paper and draw what you think will happen when we feed the \n",
+ "# output of the first network into the original second network. Draw the relationship between\n",
+ "# the input of the first network and the output of the second one."
+ ],
+ "metadata": {
+ "id": "dJbbh6R7NG9k"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# When you have a prediction, run this code to see if you were right\n",
+ "net12_out, *_ = shallow_1_1_3(net1_out, ReLU, n1_phi_0, n1_phi_1, n1_phi_2, n1_phi_3, n1_theta_10, n1_theta_11, n1_theta_20, n1_theta_21, n1_theta_30, n1_theta_31)\n",
+ "plot_neural_two_components(x, net1_out, net2_out_new, net12_out)"
+ ],
+ "metadata": {
+ "id": "BiZZl3yNM2Bq"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO \n",
+ "# Contemplate what you think will happen when we feed the \n",
+ "# output of the original first network into a second copy of the original first network, and then \n",
+ "# the output of that into the original second network (so now we have a three layer network)\n",
+ "# How many total linear regions will we have in the output? \n",
+ "net123_out, *_ = shallow_1_1_3(net12_out, ReLU, n2_phi_0, n2_phi_1, n2_phi_2, n2_phi_3, n2_theta_10, n2_theta_11, n2_theta_20, n2_theta_21, n2_theta_30, n2_theta_31)\n",
+ "plot_neural_two_components(x, net12_out, net2_out, net123_out)"
+ ],
+ "metadata": {
+ "id": "BSd51AkzNf7-"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TO DO\n",
+ "# How many linear regions would there be if we ran N copies of the first network, feeding the result of the first \n",
+ "# into the second, the second into the third and so on, and then passed the result into the original second\n",
+ "# network (blue curve above)\n",
+ "\n",
+ "# Take away conclusions: with very few parameters, we can make A LOT of linear regions, but\n",
+ "# they depend on one another in complex ways that quickly become to difficult to understand intuitively."
+ ],
+ "metadata": {
+ "id": "HqzePCLOVQK7"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
diff --git a/CM20315/CM20315_Deep2.ipynb b/CM20315/CM20315_Deep2.ipynb
new file mode 100644
index 0000000..31df371
--- /dev/null
+++ b/CM20315/CM20315_Deep2.ipynb
@@ -0,0 +1,317 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "collapsed_sections": [],
+ "authorship_tag": "ABX9TyP87B9tfgXpVQdlQBUGw4mg",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Deep neural networks #2**\n",
+ "\n",
+ "In this notebook, we'll investigate converting neural networks to matrix form."
+ ],
+ "metadata": {
+ "id": "MaKn8CFlzN8E"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "8ClURpZQzI6L"
+ },
+ "outputs": [],
+ "source": [
+ "# Imports math library\n",
+ "import numpy as np\n",
+ "# Imports plotting library\n",
+ "import matplotlib.pyplot as plt"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the Rectified Linear Unit (ReLU) function\n",
+ "def ReLU(preactivation):\n",
+ " activation = preactivation.clip(0.0)\n",
+ " return activation"
+ ],
+ "metadata": {
+ "id": "YdmveeAUz4YG"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define a shallow neural network with, one input, one output, and three hidden units\n",
+ "def shallow_1_1_3(x, activation_fn, phi_0,phi_1,phi_2,phi_3, theta_10, theta_11, theta_20, theta_21, theta_30, theta_31):\n",
+ " # Initial lines\n",
+ " pre_1 = theta_10 + theta_11 * x\n",
+ " pre_2 = theta_20 + theta_21 * x\n",
+ " pre_3 = theta_30 + theta_31 * x\n",
+ " # Activation functions\n",
+ " act_1 = activation_fn(pre_1)\n",
+ " act_2 = activation_fn(pre_2)\n",
+ " act_3 = activation_fn(pre_3)\n",
+ " # Weight activations\n",
+ " w_act_1 = phi_1 * act_1\n",
+ " w_act_2 = phi_2 * act_2\n",
+ " w_act_3 = phi_3 * act_3\n",
+ " # Combine weighted activation and add y offset\n",
+ " y = phi_0 + w_act_1 + w_act_2 + w_act_3\n",
+ " # Return everything we have calculated\n",
+ " return y, pre_1, pre_2, pre_3, act_1, act_2, act_3, w_act_1, w_act_2, w_act_3"
+ ],
+ "metadata": {
+ "id": "ximCLwIfz8kj"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# # Plot the shallow neural network. We'll assume input in is range [-1,1] and output [-1,1]\n",
+ "def plot_neural(x, y):\n",
+ " fig, ax = plt.subplots()\n",
+ " ax.plot(x.T,y.T)\n",
+ " ax.set_xlabel('Input'); ax.set_ylabel('Output')\n",
+ " ax.set_xlim([-1,1]);ax.set_ylim([-1,1])\n",
+ " ax.set_aspect(1.0)\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "btrt7BX20gKD"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's define a networks. We'll just consider the inputs and outputs over the range [-1,1]. If you set the \"plot_all\" flat to True, you can see the details of how it was created."
+ ],
+ "metadata": {
+ "id": "LxBJCObC-NTY"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now lets define some parameters and run the first neural network\n",
+ "n1_theta_10 = 0.0 ; n1_theta_11 = -1.0\n",
+ "n1_theta_20 = 0 ; n1_theta_21 = 1.0\n",
+ "n1_theta_30 = -0.67 ; n1_theta_31 = 1.0\n",
+ "n1_phi_0 = 1.0; n1_phi_1 = -2.0; n1_phi_2 = -3.0; n1_phi_3 = 9.3\n",
+ "\n",
+ "# Define a range of input values\n",
+ "n1_in = np.arange(-1,1,0.01).reshape([1,-1])\n",
+ "\n",
+ "# We run the neural network for each of these input values\n",
+ "n1_out, *_ = shallow_1_1_3(n1_in, ReLU, n1_phi_0, n1_phi_1, n1_phi_2, n1_phi_3, n1_theta_10, n1_theta_11, n1_theta_20, n1_theta_21, n1_theta_30, n1_theta_31)\n",
+ "# And then plot it\n",
+ "plot_neural(n1_in, n1_out)"
+ ],
+ "metadata": {
+ "id": "JRebvurv22pT"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now we'll define the same neural network, but this time, we will use matrix form. When you get this right, it will draw the same plot as above."
+ ],
+ "metadata": {
+ "id": "XCJqo_AjfAra"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "beta_0 = np.zeros((3,1))\n",
+ "Omega_0 = np.zeros((3,1))\n",
+ "beta_1 = np.zeros((1,1))\n",
+ "Omega_1 = np.zeros((1,3))\n",
+ "\n",
+ "# TODO Fill in the values of the beta and Omega matrices with the n1_theta and n1_phi parameters that define the network above\n",
+ "# !!! NOTE THAT MATRICES ARE CONVENTIONALLY INDEXED WITH a_11 IN THE TOP LEFT CORNER, BUT NDARRAYS START AT [0,0]\n",
+ "# To get you started I've filled in a couple:\n",
+ "beta_0[0,0] = n1_theta_10\n",
+ "Omega_0[0,0] = n1_theta_11\n",
+ "\n",
+ "\n",
+ "# Make sure that input data matrix has different inputs in its columns\n",
+ "n_data = n1_in.size\n",
+ "n_dim_in = 1\n",
+ "n1_in_mat = np.reshape(n1_in,(n_dim_in,n_data))\n",
+ "\n",
+ "# This runs the network for ALL of the inputs, x at once so we can draw graph\n",
+ "h1 = ReLU(np.matmul(beta_0,np.ones((1,n_data))) + np.matmul(Omega_0,n1_in_mat))\n",
+ "n1_out = np.matmul(beta_1,np.ones((1,n_data))) + np.matmul(Omega_1,h1)\n",
+ "\n",
+ "# Draw the network and check that it looks the same as the non-matrix case\n",
+ "plot_neural(n1_in, n1_out)"
+ ],
+ "metadata": {
+ "id": "MR0AecZYfACR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now we'll feed the output of the first network into the second one."
+ ],
+ "metadata": {
+ "id": "qOcj2Rof-o20"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now lets define some parameters and run the second neural network\n",
+ "n2_theta_10 = -0.6 ; n2_theta_11 = -1.0\n",
+ "n2_theta_20 = 0.2 ; n2_theta_21 = 1.0\n",
+ "n2_theta_30 = -0.5 ; n2_theta_31 = 1.0\n",
+ "n2_phi_0 = 0.5; n2_phi_1 = -1.0; n2_phi_2 = -1.5; n2_phi_3 = 2.0\n",
+ "\n",
+ "# Define a range of input values\n",
+ "n2_in = np.arange(-1,1,0.01)\n",
+ "\n",
+ "# We run the second neural network on the output of the first network\n",
+ "n2_out, *_ = \\\n",
+ " shallow_1_1_3(n1_out, ReLU, n2_phi_0, n2_phi_1, n2_phi_2, n2_phi_3, n2_theta_10, n2_theta_11, n2_theta_20, n2_theta_21, n2_theta_30, n2_theta_31)\n",
+ "# And then plot it\n",
+ "plot_neural(n1_in, n2_out)"
+ ],
+ "metadata": {
+ "id": "ZRjWu8i9239X"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "beta_0 = np.zeros((3,1))\n",
+ "Omega_0 = np.zeros((3,1))\n",
+ "beta_1 = np.zeros((3,1))\n",
+ "Omega_1 = np.zeros((3,3))\n",
+ "beta_2 = np.zeros((1,1))\n",
+ "Omega_2 = np.zeros((1,3))\n",
+ "\n",
+ "# TODO Fill in the values of the beta and Omega matrices for with the n1_theta, n1_phi, n2_theta, and n2_phi parameters \n",
+ "# that define the composition of the two networks above (see eqn 4.5 for Omega1 and beta1 albeit in different notation)\n",
+ "# !!! NOTE THAT MATRICES ARE CONVENTIONALLY INDEXED WITH a_11 IN THE TOP LEFT CORNER, BUT NDARRAYS START AT [0,0] SO EVERYTHING IS OFFSET\n",
+ "# To get you started I've filled in a few:\n",
+ "beta_0[0,0] = n1_theta_10\n",
+ "Omega_0[0,0] = n1_theta_11\n",
+ "beta_1[0,0] = n2_theta_10 + n2_theta_11 * n1_phi_0\n",
+ "Omega_1[0,0] = n2_theta_11 * n1_phi_1\n",
+ "\n",
+ "\n",
+ "\n",
+ "# Make sure that input data matrix has different inputs in its columns\n",
+ "n_data = n1_in.size\n",
+ "n_dim_in = 1\n",
+ "n1_in_mat = np.reshape(n1_in,(n_dim_in,n_data))\n",
+ "\n",
+ "# This runs the network for ALL of the inputs, x at once so we can draw graph (hence extra np.ones term)\n",
+ "h1 = ReLU(np.matmul(beta_0,np.ones((1,n_data))) + np.matmul(Omega_0,n1_in_mat))\n",
+ "h2 = ReLU(np.matmul(beta_1,np.ones((1,n_data))) + np.matmul(Omega_1,h1))\n",
+ "n1_out = np.matmul(beta_2,np.ones((1,n_data))) + np.matmul(Omega_2,h2)\n",
+ "\n",
+ "# Draw the network and check that it looks the same as the non-matrix version\n",
+ "plot_neural(n1_in, n1_out)"
+ ],
+ "metadata": {
+ "id": "ZB2HTalOE40X"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's make a deep network with 3 hidden layers. It will have d_i=4 inputs, d_1=5 neurons in the first layer, d_2=2 neurons in the second layer and d_3=4 neurons in the third layer, and d_o = 1 output. Consults figure 4.6 for guidance."
+ ],
+ "metadata": {
+ "id": "0VANqxH2kyS4"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# define sizes\n",
+ "D_i=4; D_1=5; D_2=2; D_3=4; D_o=1\n",
+ "# We'll choose the inputs and parameters of this network randomly using np.random.normal\n",
+ "# For example, we'll set the input using\n",
+ "n_data = 10;\n",
+ "x = np.random.normal(size=(D_i, n_data))\n",
+ "# TODO initialize the parameters randomly but with the correct sizes\n",
+ "# Replace the lines below\n",
+ "beta_0 = np.random.normal(size=(1,1))\n",
+ "Omega_0 = np.random.normal(size=(1,1))\n",
+ "beta_1 = np.random.normal(size=(1,1))\n",
+ "Omega_1 = np.random.normal(size=(1,1))\n",
+ "beta_2 = np.random.normal(size=(1,1))\n",
+ "Omega_2 = np.random.normal(size=(1,1))\n",
+ "beta_3 = np.random.normal(size=(1,1))\n",
+ "Omega_3 = np.random.normal(size=(1,1))\n",
+ "\n",
+ "# If you set the above sizes to the correct values then, the following code will run \n",
+ "h1 = ReLU(np.matmul(beta_0,np.ones((1,n_data))) + np.matmul(Omega_0,x));\n",
+ "h2 = ReLU(np.matmul(beta_1,np.ones((1,n_data))) + np.matmul(Omega_1,h1));\n",
+ "h3 = ReLU(np.matmul(beta_2,np.ones((1,n_data))) + np.matmul(Omega_2,h2));\n",
+ "y = np.matmul(beta_3,np.ones((1,n_data))) + np.matmul(Omega_3,h3)\n",
+ "\n",
+ "if h1.shape[0] is not D_1 or h1.shape[1] is not n_data:\n",
+ " print(\"h1 is wrong shape\")\n",
+ "if h2.shape[0] is not D_2 or h1.shape[1] is not n_data:\n",
+ " print(\"h2 is wrong shape\")\n",
+ "if h3.shape[0] is not D_3 or h1.shape[1] is not n_data:\n",
+ " print(\"h3 is wrong shape\")\n",
+ "if y.shape[0] is not D_o or h1.shape[1] is not n_data:\n",
+ " print(\"Output is wrong shape\")\n",
+ "\n",
+ "# Print the inputs and outputs\n",
+ "print(x)\n",
+ "print(y)"
+ ],
+ "metadata": {
+ "id": "RdBVAc_Rj22-"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
diff --git a/CM20315/CM20315_Gradients_I.ipynb b/CM20315/CM20315_Gradients_I.ipynb
new file mode 100644
index 0000000..3d15e80
--- /dev/null
+++ b/CM20315/CM20315_Gradients_I.ipynb
@@ -0,0 +1,420 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "collapsed_sections": [],
+ "authorship_tag": "ABX9TyMDEfAZvjcjpvBNmdrYv3EW",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# CM20315 Gradients I\n",
+ "\n",
+ "We're going to investigate how to take the derivatives of functions where one operation is composed with another, which is composed with a third and so on. For example, consider the function:\n",
+ "\n",
+ "\\begin{equation}\n",
+ " y = \\beta_4+\\omega_4\\cdot \\log\\biggl[\\beta_3+\\omega_3\\cdot\\cos\\Bigl[\\beta_2+\\omega_2\\cdot\\exp\\bigl[\\beta_1+\\omega_1\\cdot\\sin[\\beta_0+\\omega_0x]\\bigr]\\Bigr]\\biggr],\n",
+ "\\end{equation}\n",
+ "\n",
+ "which is a composition of the functions $\\log[\\bullet], \\cos[\\bullet],\\exp[\\bullet],\\sin[\\bullet]$. I chose these just because you probably already know the derivatives of these functions:\n",
+ "\n",
+ "\\begin{eqnarray*}\n",
+ "\\frac{\\partial \\log[z]}{\\partial z} = \\frac{1}{z}\\quad\\quad \\frac{\\partial \\cos[z]}{\\partial z} = -\\sin[z] \\quad\\quad \\frac{\\partial \\exp[z]}{\\partial z} = \\exp[z] \\quad\\quad \\frac{\\partial \\sin[z]}{\\partial z} = -\\cos[z].\n",
+ "\\end{eqnarray*}\n",
+ "\n",
+ "Suppose that we know the current values of $\\beta_{0},\\beta_{1},\\beta_{2},\\beta_{3},\\beta_{4},\\omega_{0},\\omega_{1},\\omega_{2},\\omega_{3},\\omega_{4}$, and $x$. We could obviously calculate $y$. But we also want to know how $y$ changes when we make a small change to $\\beta_{0},\\beta_{1},\\beta_{2},\\beta_{3},\\beta_{4},\\omega_{0},\\omega_{1},\\omega_{2},\\omega_{3}$, or $\\omega_{4}$. In other words, we want to compute the ten derivatives:\n",
+ "\n",
+ "\\begin{eqnarray*}\n",
+ "\\frac{\\partial y}{\\partial \\beta_{0}}, \\quad \\frac{\\partial y}{\\partial \\beta_{1}}, \\quad \\frac{\\partial y}{\\partial \\beta_{2}}, \\quad \\frac{\\partial y }{\\partial \\beta_{3}}, \\quad\n",
+ "\\frac{\\partial y}{\\partial \\beta_{4}}, \\quad \\frac{\\partial y}{\\partial \\omega_{0}}, \\quad \\frac{\\partial y}{\\partial \\omega_{1}}, \\quad \\frac{\\partial y}{\\partial \\omega_{2}}, \\quad \\frac{\\partial y}{\\partial \\omega_{3}}, \\quad\\mbox{and} \\quad \\frac{\\partial y}{\\partial \\omega_{4}}.\n",
+ "\\end{eqnarray*}"
+ ],
+ "metadata": {
+ "id": "1DmMo2w63CmT"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# import library\n",
+ "import numpy as np"
+ ],
+ "metadata": {
+ "id": "RIPaoVN834Lj"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's first define the original function for $y$:"
+ ],
+ "metadata": {
+ "id": "32-ufWhc3v2c"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "AakK_qen3BpU"
+ },
+ "outputs": [],
+ "source": [
+ "def fn(x, beta0, beta1, beta2, beta3, beta4, omega0, omega1, omega2, omega3, omega4):\n",
+ " return beta4 + omega4 * np.log(beta3+omega3 * np.cos(beta2 + omega2 * np.exp(beta1 + omega1 * np.sin(beta0 + omega0 * x))))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now we'll choose some values for the betas and the omegas and x and compute the output of the function:"
+ ],
+ "metadata": {
+ "id": "y7tf0ZMt5OXt"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "beta0 = 1.0; beta1 = 2.0; beta2 = -3.0; beta3 = 0.4; beta4 = -0.3\n",
+ "omega0 = 0.1; omega1 = -0.4; omega2 = 2.0; omega3 = 3.0; omega4 = -0.5\n",
+ "x = 2.3\n",
+ "y_func = fn(x,beta0,beta1,beta2,beta3,beta4,omega0,omega1,omega2,omega3,omega4)\n",
+ "print('y=%3.3f'%y_func)"
+ ],
+ "metadata": {
+ "id": "pwvOcCxr41X_"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Computing derivatives by hand\n",
+ "\n",
+ "We could compute expressions for the derivatives by hand and write code to compute them directly. Some of them are easy. For example:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\frac{\\partial y}{\\partial \\beta_{4}} = 1,\n",
+ "\\end{equation}\n",
+ "\n",
+ "but some have very complex expressions, even for this relatively simple original equation. For example:\n",
+ "\n",
+ "\\begin{eqnarray*}\n",
+ "\\frac{\\partial y}{\\partial \\omega_{0}} &=& \n",
+ "-\\frac{\\omega_{1}\\omega_{2}\\omega_{3}\\omega_{4} x \\cos[\\beta_{0}\\!+\\!\\omega_{0}x]\\cdot\\exp\\bigl[\\omega_{1}\\sin[\\beta_{0}\\!+\\!\\omega_{0}x]\\!+\\!\\beta_{1}\\bigr]\\cdot\\sin\\Bigl[\\omega_{2}\\exp\\bigl[\\omega_{1}\\sin[\\beta_{0}\\!+\\!\\omega_{0}x]\\!+\\!\\beta_{1}\\bigr]\\!+\\!\\beta_{2}\\Bigr]}\n",
+ "{\\omega_{3}\\cos[\\omega_{2}\\exp[\\omega_{1}\\sin[\\beta_{0}\\!+\\!\\omega_{0}x]\\!+\\!\\beta_{1}]\\!+\\!\\beta_{2}]\\!+\\!\\beta_{3}}.\n",
+ "\\end{eqnarray*}"
+ ],
+ "metadata": {
+ "id": "u5w69NeT64yV"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "dydbeta4_func = 1\n",
+ "dydomega0_func = -omega1*omega2*omega3*omega4*x * np.cos(beta0+omega0*x) * \\\n",
+ " np.exp(omega1 * np.sin(beta0+omega0*x)+beta1) * \\\n",
+ " np.sin(omega2 * np.exp(omega1 * np.sin(beta0+omega0 *x)+beta1)+beta2)/ \\\n",
+ " (omega3 * np.cos(omega2 * np.exp(omega1 * np.sin(beta0+omega0*x)+beta1)+beta2)+beta3)"
+ ],
+ "metadata": {
+ "id": "7t22hALp5zkq"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's make sure these are correct using finite differences:"
+ ],
+ "metadata": {
+ "id": "iRh4hnu3-H3n"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "dydbeta4_fd = (fn(x,beta0,beta1,beta2,beta3,beta4+0.0001,omega0,omega1,omega2,omega3,omega4)-fn(x,beta0,beta1,beta2,beta3,beta4,omega0,omega1,omega2,omega3,omega4))/0.0001\n",
+ "dydomega0_fd = (fn(x,beta0,beta1,beta2,beta3,beta4,omega0+0.0001,omega1,omega2,omega3,omega4)-fn(x,beta0,beta1,beta2,beta3,beta4,omega0,omega1,omega2,omega3,omega4))/0.0001\n",
+ "\n",
+ "print('dydbeta4: Function value = %3.3f, Finite difference value = %3.3f'%(dydbeta4_func,dydbeta4_fd))\n",
+ "print('dydomega0: Function value = %3.3f, Finite difference value = %3.3f'%(dydomega0_func,dydomega0_fd))"
+ ],
+ "metadata": {
+ "id": "1O3XmXMx-HlZ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The code to calculate $\\partial y/ \\partial \\omega_0$ is a bit of a nightmare. It's easy to make mistakes, and you can see that some parts of it are repeated (for example, the $\\sin[\\bullet]$ term), which suggests some kind of redundancy in the calculations. The goal of this practical is to compute the derivatives in a much simpler way. There will be three steps:"
+ ],
+ "metadata": {
+ "id": "wS4IPjZAKWTN"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "**Step 1:** Write the original equations as a series of intermediate calculations. We change \n",
+ "\n",
+ "\\begin{equation}\n",
+ " y = \\beta_4+\\omega_4\\cdot \\log\\biggl[\\beta_3+\\omega_3\\cdot\\cos\\Bigl[\\beta_2+\\omega_2\\cdot\\exp\\bigl[\\beta_1+\\omega_1\\cdot\\sin[\\beta_0+\\omega_0x]\\bigr]\\Bigr]\\biggr]\n",
+ "\\end{equation}\n",
+ "\n",
+ "to \n",
+ "\n",
+ "\\begin{eqnarray}\n",
+ "f_{0} &=& \\beta_{0} + \\omega_{0} x\\nonumber\\\\\n",
+ "h_{1} &=& \\sin[f_{0}]\\nonumber\\\\\n",
+ "f_{1} &=& \\beta_{1} + \\omega_{1}h_{1}\\nonumber\\\\\n",
+ "h_{2} &=& \\exp[f_{1}]\\nonumber\\\\\n",
+ "f_{2} &=& \\beta_{2} + \\omega_{2} h_{2}\\nonumber\\\\\n",
+ "h_{3} &=& \\cos[f_{2}]\\nonumber\\\\\n",
+ "f_{3} &=& \\beta_{3} + \\omega_{3}h_{3}\\nonumber\\\\\n",
+ "h_{4} &=& \\log[f_{3}]\\nonumber\\\\\n",
+ "y &=& \\beta_{4} + \\omega_{4} h_{4}\n",
+ "\\end{eqnarray}\n",
+ "\n",
+ "and compute and store the values of all of these intermediate values. We'll need them to compute the derivatives."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Gradients II: Backpropagation algorithm\n",
+ "\n",
+ "In this practical, we'll investigate the backpropagation algorithm. This computes the gradients of the loss with respect to all of the parameters (weights and biases) in the network. We'll use these gradients when we run stochastic gradient descent."
+ ],
+ "metadata": {
+ "id": "L6chybAVFJW2"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "LdIDglk1FFcG"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "First let's define a neural network. We'll just choose the weights and biases randomly for now"
+ ],
+ "metadata": {
+ "id": "nnUoI0m6GyjC"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set seed so we always get the same random numbers\n",
+ "np.random.seed(0)\n",
+ "\n",
+ "# Number of layers\n",
+ "K = 5\n",
+ "# Number of neurons per layer\n",
+ "D = 6\n",
+ "# Input layer\n",
+ "D_i = 1\n",
+ "# Output layer \n",
+ "D_o = 1\n",
+ "\n",
+ "# Make empty lists \n",
+ "all_weights = [None] * (K+1)\n",
+ "all_biases = [None] * (K+1)\n",
+ "\n",
+ "# Create input and output layers\n",
+ "all_weights[0] = np.random.normal(size=(D, D_i))\n",
+ "all_weights[-1] = np.random.normal(size=(D_o, D))\n",
+ "all_biases[0] = np.random.normal(size =(D,1))\n",
+ "all_biases[-1]= np.random.normal(size =(D_o,1))\n",
+ "\n",
+ "# Create intermediate layers\n",
+ "for layer in range(1,K):\n",
+ " all_weights[layer] = np.random.normal(size=(D,D))\n",
+ " all_biases[layer] = np.random.normal(size=(D,1)) "
+ ],
+ "metadata": {
+ "id": "WVM4Tc_jGI0Q"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the Rectified Linear Unit (ReLU) function\n",
+ "def ReLU(preactivation):\n",
+ " activation = preactivation.clip(0.0)\n",
+ " return activation"
+ ],
+ "metadata": {
+ "id": "jZh-7bPXIDq4"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's run our random network. The weight matrices $\\boldsymbol\\Omega_{1\\ldots K}$ are the entries of the list \"all_weights\" and the biases $\\boldsymbol\\beta_{1\\ldots k}$ are the entries of the list \"all_biases\"\n",
+ "\n",
+ "We know that we will need the activations $\\mathbf{f}_{0\\ldots K}$ and the activations $\\mathbf{h}_{1\\ldots K}$ for the forward pass of backpropagation, so we'll store and return these as well. \n"
+ ],
+ "metadata": {
+ "id": "5irtyxnLJSGX"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def compute_network_output(net_input, all_weights, all_biases):\n",
+ "\n",
+ " # Retrieve number of layers\n",
+ " K = len(all_weights) -1\n",
+ "\n",
+ " # We'll store the pre-activations at each layer in a list \"all_f\"\n",
+ " # and the activations in a second list[all_h]. \n",
+ " all_f = [None] * (K+1)\n",
+ " all_h = [None] * (K+1)\n",
+ "\n",
+ " #For convenience, we'll set \n",
+ " # all_h[0] to be the input, and all_f[K] will be the output\n",
+ " all_h[0] = net_input\n",
+ "\n",
+ " # Run through the layers, calculating all_f[0...K-1] and all_h[1...K]\n",
+ " for layer in range(K):\n",
+ " # Update preactivations and activations at this layer according to eqn 7.5\n",
+ " # Remmember to use np.matmul for matrix multiplications\n",
+ " # TODO -- Replace the lines below\n",
+ " all_f[layer] = all_h[layer]\n",
+ " all_h[layer+1] = all_f[layer]\n",
+ "\n",
+ " # Compute the output from the last hidden layer\n",
+ " # TO DO -- Replace the line below\n",
+ " all_f[K] = np.zeros_like(all_biases[-1])\n",
+ "\n",
+ " # Retrieve the output\n",
+ " net_output = all_f[K]\n",
+ "\n",
+ " return net_output, all_f, all_h"
+ ],
+ "metadata": {
+ "id": "LgquJUJvJPaN"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define in input\n",
+ "net_input = np.ones((D_i,1)) * 1.2\n",
+ "# Compute network output\n",
+ "net_output, all_f, all_h = compute_network_output(net_input,all_weights, all_biases)\n",
+ "print(\"True output = %3.3f, Your answer = %3.3f\"%(1.907, net_output[0,0]))"
+ ],
+ "metadata": {
+ "id": "IN6w5m2ZOhnB"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's define a loss function. We'll just use the least squares loss function. We'll also write a function to compute dloss_doutpu"
+ ],
+ "metadata": {
+ "id": "SxVTKp3IcoBF"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def least_squares_loss(net_output, y):\n",
+ " return np.sum((net_output-y) * (net_output-y))\n",
+ "\n",
+ "def d_loss_d_output(net_output, y):\n",
+ " return 2*(net_output -y); "
+ ],
+ "metadata": {
+ "id": "6XqWSYWJdhQR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "y = np.ones((D_o,1)) * 20.0\n",
+ "loss = least_squares_loss(net_output, y)\n",
+ "print(\"y = %3.3f Loss = %3.3f\"%(y, loss))"
+ ],
+ "metadata": {
+ "id": "njF2DUQmfttR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's compute the derivatives of the network. We already computed the forward pass. Let's compute the backward pass."
+ ],
+ "metadata": {
+ "id": "98WmyqFYWA-0"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# We'll need the indicator function\n",
+ "def indicator_function(x):\n",
+ " x_in = np.array(x)\n",
+ " x_in[x_in>=0] = 1\n",
+ " x_in[x_in<0] = 0\n",
+ " return x_in\n",
+ "\n",
+ "# Main backward pass routine\n",
+ "def backward_pass(all_weights, all_biases, all_f, all_h, y):\n",
+ " # We'll store the derivatives dl_dweights and dl_dbiases in lists as well\n",
+ " all_dl_dweights = [None] * (K+1)\n",
+ " all_dl_dbiases = [None] * (K+1)\n",
+ " # And we'll store the derivatives of the loss with respect to the activation and preactivations in lists\n",
+ " all_dl_df = [None] * (K+1)\n",
+ " all_dl_dh = [None] * (K+1)\n",
+ " # Again for convenience we'll stick with the convention that all_h[0] is the net input and all_f[k] in the net output\n",
+ "\n",
+ " # Compute derivatives of net output with respect to loss\n",
+ " all_dl_df[K] = np.array(d_loss_d_output(all_f[K],y))\n",
+ "\n",
+ " # Now work backwards through the network\n",
+ " for layer in range(K,-1,-1):\n",
+ " # TODO Calculate the derivatives of biases at layer from all_dl_df[K]. (eq 7.13, line 1)\n",
+ " # NOTE! To take a copy of matrix X, use Z=np.array(X)\n",
+ " # REPLACE THIS LINE\n",
+ " all_dl_dbiases[layer] = np.zeros_like(all_biases[layer])\n",
+ "\n",
+ " # TODO Calculate the derivatives of weight at layer from all_dl_df[K] and all_h[K] (eq 7.13, line 2)\n",
+ " # Don't forget to use np.matmul\n",
+ " # REPLACE THIS LINE\n",
+ " all_dl_dweights[layer] = np.zeros_like(all_weights[layer])\n",
+ "\n",
+ " # TODO: calculate the derivatives of activations from weight and derivatives of next preactivations (eq 7.13, line 3 second part)\n",
+ " # REPLACE THIS LINE\n",
+ " all_dl_dh[layer] = np.zeros_like(all_h[layer])\n",
+ "\n",
+ " if layer > 0:\n",
+ " # TODO Calculate the derivatives of the pre-activation f with respect to activation h (eq 7.13, line 3, first part)\n",
+ " # REPLACE THIS LINE\n",
+ " all_dl_df[layer-1] = np.zeros_like(all_f[layer-1])\n",
+ "\n",
+ " return all_dl_dweights, all_dl_dbiases"
+ ],
+ "metadata": {
+ "id": "LJng7WpRPLMz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "all_dl_dweights, all_dl_dbiases = backward_pass(all_weights, all_biases, all_f, all_h, y)"
+ ],
+ "metadata": {
+ "id": "9A9MHc4sQvbp"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "np.set_printoptions(precision=3)\n",
+ "# Make space for derivatives computed by finite differences\n",
+ "all_dl_dweights_fd = [None] * (K+1)\n",
+ "all_dl_dbiases_fd = [None] * (K+1)\n",
+ "\n",
+ "# Let's test if we have the derivatives right using finite differences\n",
+ "delta_fd = 0.000001\n",
+ "\n",
+ "# For every layer\n",
+ "for layer in range(K):\n",
+ " dl_dbias = np.zeros_like(all_dl_dbiases[layer])\n",
+ " # For every element in the bias\n",
+ " for row in range(all_biases[layer].shape[0]):\n",
+ " # Take copy of biases We'll change one element each time\n",
+ " all_biases_copy = [np.array(x) for x in all_biases]\n",
+ " all_biases_copy[layer][row] += delta_fd\n",
+ " network_output_1, *_ = compute_network_output(net_input, all_weights, all_biases_copy)\n",
+ " network_output_2, *_ = compute_network_output(net_input, all_weights, all_biases)\n",
+ " dl_dbias[row] = (least_squares_loss(network_output_1, y) - least_squares_loss(network_output_2,y))/delta_fd\n",
+ " all_dl_dbiases_fd[layer] = np.array(dl_dbias)\n",
+ " print(\"Bias %d, derivatives from backprop:\"%(layer))\n",
+ " print(all_dl_dbiases[layer])\n",
+ " print(\"Bias %d, derivatives from finite differences\"%(layer))\n",
+ " print(all_dl_dbiases_fd[layer])\n",
+ "\n",
+ "\n",
+ "# For every layer\n",
+ "for layer in range(K):\n",
+ " dl_dweight = np.zeros_like(all_dl_dweights[layer])\n",
+ " # For every element in the bias\n",
+ " for row in range(all_weights[layer].shape[0]):\n",
+ " for col in range(all_weights[layer].shape[1]):\n",
+ " # Take copy of biases We'll change one element each time\n",
+ " all_weights_copy = [np.array(x) for x in all_weights]\n",
+ " all_weights_copy[layer][row][col] += delta_fd\n",
+ " network_output_1, *_ = compute_network_output(net_input, all_weights_copy, all_biases)\n",
+ " network_output_2, *_ = compute_network_output(net_input, all_weights, all_biases)\n",
+ " dl_dweight[row][col] = (least_squares_loss(network_output_1, y) - least_squares_loss(network_output_2,y))/delta_fd\n",
+ " all_dl_dweights_fd[layer] = np.array(dl_dweight)\n",
+ " print(\"Weight %d, derivatives from backprop:\"%(layer))\n",
+ " print(all_dl_dweights[layer])\n",
+ " print(\"Weight %d, derivatives from finite differences\"%(layer))\n",
+ " print(all_dl_dweights_fd[layer])"
+ ],
+ "metadata": {
+ "id": "PK-UtE3hreAK"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "gtokc0VX0839"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
diff --git a/CM20315/CM20315_Gradients_III.ipynb b/CM20315/CM20315_Gradients_III.ipynb
new file mode 100644
index 0000000..e691d04
--- /dev/null
+++ b/CM20315/CM20315_Gradients_III.ipynb
@@ -0,0 +1,351 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "collapsed_sections": [],
+ "authorship_tag": "ABX9TyPr1jNETAJLP27xFPVEC09J",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Initialization\n",
+ "\n",
+ "In this practical, we'll investigate the what happens to the activations and the forward pass if we don't initialize the parameters sensibly."
+ ],
+ "metadata": {
+ "id": "L6chybAVFJW2"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "LdIDglk1FFcG"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "First let's define a neural network. We'll just choose the weights and biases randomly for now"
+ ],
+ "metadata": {
+ "id": "nnUoI0m6GyjC"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def init_params(K, D, sigma_sq_omega):\n",
+ " # Set seed so we always get the same random numbers\n",
+ " np.random.seed(0)\n",
+ "\n",
+ " # Input layer\n",
+ " D_i = 1\n",
+ " # Output layer \n",
+ " D_o = 1\n",
+ "\n",
+ " # Make empty lists \n",
+ " all_weights = [None] * (K+1)\n",
+ " all_biases = [None] * (K+1)\n",
+ "\n",
+ " # Create input and output layers\n",
+ " all_weights[0] = np.random.normal(size=(D, D_i))*np.sqrt(sigma_sq_omega)\n",
+ " all_weights[-1] = np.random.normal(size=(D_o, D)) * np.sqrt(sigma_sq_omega)\n",
+ " all_biases[0] = np.zeros((D,1))\n",
+ " all_biases[-1]= np.zeros((D_o,1))\n",
+ "\n",
+ " # Create intermediate layers\n",
+ " for layer in range(1,K):\n",
+ " all_weights[layer] = np.random.normal(size=(D,D))*np.sqrt(sigma_sq_omega)\n",
+ " all_biases[layer] = np.zeros((D,1)) \n",
+ "\n",
+ " return all_weights, all_biases"
+ ],
+ "metadata": {
+ "id": "WVM4Tc_jGI0Q"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the Rectified Linear Unit (ReLU) function\n",
+ "def ReLU(preactivation):\n",
+ " activation = preactivation.clip(0.0)\n",
+ " return activation"
+ ],
+ "metadata": {
+ "id": "jZh-7bPXIDq4"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def compute_network_output(net_input, all_weights, all_biases):\n",
+ "\n",
+ " # Retrieve number of layers\n",
+ " K = len(all_weights) -1\n",
+ "\n",
+ " # We'll store the pre-activations at each layer in a list \"all_f\"\n",
+ " # and the activations in a second list[all_h]. \n",
+ " all_f = [None] * (K+1)\n",
+ " all_h = [None] * (K+1)\n",
+ "\n",
+ " #For convenience, we'll set \n",
+ " # all_h[0] to be the input, and all_f[K] will be the output\n",
+ " all_h[0] = net_input\n",
+ "\n",
+ " # Run through the layers, calculating all_f[0...K-1] and all_h[1...K]\n",
+ " for layer in range(K):\n",
+ " # Update preactivations and activations at this layer according to eqn 7.5\n",
+ " all_f[layer] = all_biases[layer] + np.matmul(all_weights[layer], all_h[layer])\n",
+ " all_h[layer+1] = ReLU(all_f[layer])\n",
+ "\n",
+ " # Compute the output from the last hidden layer\n",
+ " all_f[K] = all_biases[K] + np.matmul(all_weights[K], all_h[K])\n",
+ "\n",
+ " # Retrieve the output\n",
+ " net_output = all_f[K]\n",
+ "\n",
+ " return net_output, all_f, all_h"
+ ],
+ "metadata": {
+ "id": "LgquJUJvJPaN"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's investigate how this the size of the outputs vary as we change the initialization variance:\n"
+ ],
+ "metadata": {
+ "id": "bIUrcXnOqChl"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Number of layers\n",
+ "K = 5\n",
+ "# Number of neurons per layer\n",
+ "D = 8\n",
+ " # Input layer\n",
+ "D_i = 1\n",
+ "# Output layer \n",
+ "D_o = 1\n",
+ "# Set variance of initial weights to 1\n",
+ "sigma_sq_omega = 1.0\n",
+ "# Initialize parameters\n",
+ "all_weights, all_biases = init_params(K,D,sigma_sq_omega)\n",
+ "\n",
+ "n_data = 1000\n",
+ "data_in = np.random.normal(size=(1,n_data))\n",
+ "net_output, all_f, all_h = compute_network_output(data_in, all_weights, all_biases)\n",
+ "\n",
+ "for layer in range(K):\n",
+ " print(\"Layer %d, std of hidden units = %3.3f\"%(layer, np.std(all_h[layer])))"
+ ],
+ "metadata": {
+ "id": "A55z3rKBqO7M"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# You can see that the values of the hidden units are increasing on average (the variance is across all hidden units at the layer \n",
+ "# and the 1000 training examples\n",
+ "\n",
+ "# TO DO \n",
+ "# Change this to 50 layers with 80 hidden units per layer\n",
+ "\n",
+ "# TO DO \n",
+ "# Now experiment with sigma_sq_omega to try to stop the variance of the forward computation explode"
+ ],
+ "metadata": {
+ "id": "VL_SO4tar3DC"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's define a loss function. We'll just use the least squares loss function. We'll also write a function to compute dloss_doutput\n"
+ ],
+ "metadata": {
+ "id": "SxVTKp3IcoBF"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def least_squares_loss(net_output, y):\n",
+ " return np.sum((net_output-y) * (net_output-y))\n",
+ "\n",
+ "def d_loss_d_output(net_output, y):\n",
+ " return 2*(net_output -y); "
+ ],
+ "metadata": {
+ "id": "6XqWSYWJdhQR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Here's the code for the backward pass"
+ ],
+ "metadata": {
+ "id": "98WmyqFYWA-0"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# We'll need the indicator function\n",
+ "def indicator_function(x):\n",
+ " x_in = np.array(x)\n",
+ " x_in[x_in>=0] = 1\n",
+ " x_in[x_in<0] = 0\n",
+ " return x_in\n",
+ "\n",
+ "# Main backward pass routine\n",
+ "def backward_pass(all_weights, all_biases, all_f, all_h, y):\n",
+ " # We'll store the derivatives dl_dweights and dl_dbiases in lists as well\n",
+ " all_dl_dweights = [None] * (K+1)\n",
+ " all_dl_dbiases = [None] * (K+1)\n",
+ " # And we'll store the derivatives of the loss with respect to the activation and preactivations in lists\n",
+ " all_dl_df = [None] * (K+1)\n",
+ " all_dl_dh = [None] * (K+1)\n",
+ " # Again for convenience we'll stick with the convention that all_h[0] is the net input and all_f[k] in the net output\n",
+ "\n",
+ " # Compute derivatives of net output with respect to loss\n",
+ " all_dl_df[K] = np.array(d_loss_d_output(all_f[K],y))\n",
+ "\n",
+ " # Now work backwards through the network\n",
+ " for layer in range(K,-1,-1):\n",
+ " # Calculate the derivatives of biases at layer from all_dl_df[K]. (eq 7.13, line 1)\n",
+ " all_dl_dbiases[layer] = np.array(all_dl_df[layer])\n",
+ " # Calculate the derivatives of weight at layer from all_dl_df[K] and all_h[K] (eq 7.13, line 2)\n",
+ " all_dl_dweights[layer] = np.matmul(all_dl_df[layer], all_h[layer].transpose())\n",
+ "\n",
+ " # Calculate the derivatives of activations from weight and derivatives of next preactivations (eq 7.13, line 3 second part)\n",
+ " all_dl_dh[layer] = np.matmul(all_weights[layer].transpose(), all_dl_df[layer])\n",
+ " # Calculate the derivatives of the pre-activation f with respect to activation h (eq 7.13, line 3, first part)\n",
+ " if layer > 0:\n",
+ " all_dl_df[layer-1] = indicator_function(all_f[layer-1]) * all_dl_dh[layer]\n",
+ "\n",
+ " return all_dl_dweights, all_dl_dbiases, all_dl_dh, all_dl_df"
+ ],
+ "metadata": {
+ "id": "LJng7WpRPLMz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's look at what happens to the magnitude of the gradients on the way back."
+ ],
+ "metadata": {
+ "id": "phFnbthqwhFi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Number of layers\n",
+ "K = 5\n",
+ "# Number of neurons per layer\n",
+ "D = 8\n",
+ " # Input layer\n",
+ "D_i = 1\n",
+ "# Output layer \n",
+ "D_o = 1\n",
+ "# Set variance of initial weights to 1\n",
+ "sigma_sq_omega = 1.0\n",
+ "# Initialize parameters\n",
+ "all_weights, all_biases = init_params(K,D,sigma_sq_omega)\n",
+ "\n",
+ "# For simplicity we'll just consider the gradients of the weights and biases between the first and last hidden layer\n",
+ "n_data = 100\n",
+ "aggregate_dl_df = [None] * (K+1)\n",
+ "for layer in range(1,K):\n",
+ " # These 3D arrays will store the gradients for every data point\n",
+ " aggregate_dl_df[layer] = np.zeros((D,n_data))\n",
+ "\n",
+ "\n",
+ "# We'll have to compute the derivatives of the parameters for each data point separately\n",
+ "for c_data in range(n_data):\n",
+ " data_in = np.random.normal(size=(1,1))\n",
+ " y = np.zeros((1,1))\n",
+ " net_output, all_f, all_h = compute_network_output(data_in, all_weights, all_biases)\n",
+ " all_dl_dweights, all_dl_dbiases, all_dl_dh, all_dl_df = backward_pass(all_weights, all_biases, all_f, all_h, y)\n",
+ " for layer in range(1,K):\n",
+ " aggregate_dl_df[layer][:,c_data] = np.squeeze(all_dl_df[layer])\n",
+ "\n",
+ "for layer in range(1,K):\n",
+ " print(\"Layer %d, std of dl_dh = %3.3f\"%(layer, np.std(aggregate_dl_df[layer].ravel())))\n"
+ ],
+ "metadata": {
+ "id": "9A9MHc4sQvbp"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# You can see that the values of the hidden units are increasing on average (the variance is across all hidden units at the layer \n",
+ "# and the 1000 training examples\n",
+ "\n",
+ "# TO DO \n",
+ "# Change this to 50 layers with 80 hidden units per layer\n",
+ "\n",
+ "# TO DO \n",
+ "# Now experiment with sigma_sq_omega to try to stop the variance of the gradients exploding"
+ ],
+ "metadata": {
+ "id": "gtokc0VX0839"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
diff --git a/CM20315/CM20315_Intro.ipynb b/CM20315/CM20315_Intro.ipynb
new file mode 100644
index 0000000..ba811db
--- /dev/null
+++ b/CM20315/CM20315_Intro.ipynb
@@ -0,0 +1,298 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyMTtWwAtwIZJoIsfGehxMMC",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "\n",
+ "# **CM20315 Background Mathematics**\n",
+ "\n",
+ "The purpose of this Python notebook is to make sure you can use CoLab and to familiarize yourself with some of the background mathematical concepts that you are going to need to understand deep learning. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "\n",
+ "# **CM20315 Background Mathematics**\n",
+ "\n",
+ "The purpose of this Python notebook is to make sure you can use CoLab and to familiarize yourself with some of the background mathematical concepts that you are going to need to understand deep learning. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Loss functions\n",
+ "\n",
+ "In this practical, we'll investigate loss functions. In part 1 (this notebook), we'll investigate univariate regression (where the output data $y$ is continuous. Our formulation will be based on the normal/Gaussian distribution.\n",
+ "\n",
+ "We'll compute loss functions for maximum likelihood, minimum negative log likelihood, and least squares and show that they all imply that we should use the same parameter values\n",
+ "\n",
+ "In part II, we'll investigate binary classification (where the output data is 0 or 1). This will be based on the Bernoulli distribution\n",
+ "\n",
+ "In part III we'll investigate multiclass classification (where the output data is 0,1, or, 2). This will be based on the categorical distribution."
+ ],
+ "metadata": {
+ "id": "jSlFkICHwHQF"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "PYMZ1x-Pv1ht"
+ },
+ "outputs": [],
+ "source": [
+ "# Imports math library\n",
+ "import numpy as np\n",
+ "# Imports plotting library\n",
+ "import matplotlib.pyplot as plt\n",
+ "# Import math Library\n",
+ "import math"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the Rectified Linear Unit (ReLU) function\n",
+ "def ReLU(preactivation):\n",
+ " activation = preactivation.clip(0.0)\n",
+ " return activation\n",
+ "\n",
+ "# Define a shallow neural network\n",
+ "def shallow_nn(x, beta_0, omega_0, beta_1, omaga_1):\n",
+ " # Make sure that input data is (1 x n_data) array\n",
+ " n_data = x.size\n",
+ " x = np.reshape(x,(1,n_data))\n",
+ "\n",
+ " # This runs the network for ALL of the inputs, x at once so we can draw graph\n",
+ " h1 = ReLU(np.matmul(beta_0,np.ones((1,n_data))) + np.matmul(omega_0,x))\n",
+ " y = np.matmul(beta_1,np.ones((1,n_data))) + np.matmul(omega_1,h1)\n",
+ " return y"
+ ],
+ "metadata": {
+ "id": "Fv7SZR3tv7mV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Utility function for plotting data\n",
+ "def plot_univariate_regression(x_model, y_model, x_data = None, y_data = None, sigma_model = None, title= None):\n",
+ " # Make sure model data are 1D arrays\n",
+ " x_model = np.squeeze(x_model)\n",
+ " y_model = np.squeeze(y_model)\n",
+ "\n",
+ " fig, ax = plt.subplots()\n",
+ " ax.plot(x_model,y_model)\n",
+ " if sigma_model is not None:\n",
+ " ax.fill_between(x_model, y_model-2*sigma_model, y_model+2*sigma_model, color='lightgray')\n",
+ " ax.set_xlabel('Input, $x$'); ax.set_ylabel('Output, $y$')\n",
+ " ax.set_xlim([0,1]);ax.set_ylim([-1,1])\n",
+ " ax.set_aspect(0.5)\n",
+ " if title is not None:\n",
+ " ax.set_title(title)\n",
+ " if x_data is not None:\n",
+ " ax.plot(x_data, y_data, 'ko')\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "NRR67ri_1TzN"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Univariate regression"
+ ],
+ "metadata": {
+ "id": "PsgLZwsPxauP"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Get parameters for model -- we can call this function to easily reset them\n",
+ "def get_parameters():\n",
+ " # And we'll create a network that approximately fits it\n",
+ " beta_0 = np.zeros((3,1)); # formerly theta_x0\n",
+ " omega_0 = np.zeros((3,1)); # formerly theta_x1\n",
+ " beta_1 = np.zeros((1,1)); # formerly phi_0\n",
+ " omega_1 = np.zeros((1,3)); # formerly phi_x\n",
+ "\n",
+ " beta_0[0,0] = 0.3; beta_0[1,0] = -1.0; beta_0[2,0] = -0.5\n",
+ " omega_0[0,0] = -1.0; omega_0[1,0] = 1.8; omega_0[2,0] = 0.65\n",
+ " beta_1[0,0] = 0.1;\n",
+ " omega_1[0,0] = -2.0; omega_1[0,1] = -1.0; omega_1[0,2] = 7.0\n",
+ "\n",
+ " return beta_0, omega_0, beta_1, omega_1"
+ ],
+ "metadata": {
+ "id": "pUT9Ain_HRim"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's create some 1D training data\n",
+ "x_train = np.array([0.09291784,0.46809093,0.93089486,0.67612654,0.73441752,0.86847339,\\\n",
+ " 0.49873225,0.51083168,0.18343972,0.99380898,0.27840809,0.38028817,\\\n",
+ " 0.12055708,0.56715537,0.92005746,0.77072270,0.85278176,0.05315950,\\\n",
+ " 0.87168699,0.58858043])\n",
+ "y_train = np.array([-0.25934537,0.18195445,0.651270150,0.13921448,0.09366691,0.30567674,\\\n",
+ " 0.372291170,0.20716968,-0.08131792,0.51187806,0.16943738,0.3994327,\\\n",
+ " 0.019062570,0.55820410,0.452564960,-0.1183121,0.02957665,-1.24354444, \\\n",
+ " 0.248038840,0.26824970])\n",
+ "\n",
+ "# Get parameters for the model\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "sigma = 0.2\n",
+ "\n",
+ "# Define a range of input values\n",
+ "x_model = np.arange(0,1,0.01)\n",
+ "# Run the model to get values to plot and plot it.\n",
+ "y_model = shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
+ "plot_univariate_regression(x_model, y_model, x_train, y_train, sigma_model = sigma)\n"
+ ],
+ "metadata": {
+ "id": "VWzNOt1swFVd"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The blue line is the mean prediction of the model and the gray area represents plus/minus two standard deviations. This model fits okay, but could be improved. Let's compute the loss. We'll compute the the least squares error, the likelihood, the negative log likelihood."
+ ],
+ "metadata": {
+ "id": "MvVX6tl9AEXF"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Return probability under normal distribution for input x\n",
+ "def normal_distribution(y, mu, sigma):\n",
+ " # TODO-- write in the equation for the normal distribution \n",
+ " # Equation 5.7 from the notes (you will need np.sqrt() and np.exp(), and math.pi)\n",
+ " # Don't use the numpy version -- that's cheating!\n",
+ " # Replace the line below\n",
+ " prob = np.zeros_like(y)\n",
+ " return prob"
+ ],
+ "metadata": {
+ "id": "YaLdRlEX0FkU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's double check we get the right answer before proceeding\n",
+ "print(\"Correct answer = %3.3f, Your answer = %3.3f\"%(0.119,normal_distribution(1,-1,2.3)))"
+ ],
+ "metadata": {
+ "id": "4TSL14dqHHbV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's plot the Gaussian distribution.\n",
+ "y_gauss = np.arange(-5,5,0.1)\n",
+ "mu = 0; sigma = 1.0\n",
+ "gauss_prob = normal_distribution(y_gauss, mu, sigma)\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(y_gauss, gauss_prob)\n",
+ "ax.set_xlabel('Input, $y$'); ax.set_ylabel('Probability $Pr(y)$')\n",
+ "ax.set_xlim([-5,5]);ax.set_ylim([0,1.0])\n",
+ "plt.show()\n",
+ "\n",
+ "# TODO \n",
+ "# 1. Predict what will happen if we change to mu=1 and leave sigma=1\n",
+ "# Answer:\n",
+ "# Now change the code above and see if you were correct.\n",
+ "# 2. Predict what will happen if we leave mu = 0 and change sigma to 2.0\n",
+ "# Answer:\n",
+ "# 3. Predict what will happen if we leave mu = 0 and change sigma to 0.5\n",
+ "# Answer:"
+ ],
+ "metadata": {
+ "id": "A2HcmNfUMIlj"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's compute the likelihood using this function"
+ ],
+ "metadata": {
+ "id": "R5z_0dzQMF35"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Return the likelihood of all of the data under the model\n",
+ "def compute_likelihood(y_train, mu, sigma):\n",
+ " # TODO -- compute the likelihood of the data -- the product of the normal probabilities for each data point\n",
+ " # Top line of equation 5.3 in the notes\n",
+ " # You will need np.prod() and the normal_distribution function you used above\n",
+ " # Replace the line below\n",
+ " likelihood = 0\n",
+ " return likelihood"
+ ],
+ "metadata": {
+ "id": "zpS7o6liCx7f"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's test this for a homoscedastic (constant sigma) model\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "# Use our neural network to predict the mean of the Gaussian\n",
+ "mu_pred = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+ "# Set the standard deviation to something reasonable\n",
+ "sigma = 0.2\n",
+ "# Compute the likelihood\n",
+ "likelihood = compute_likelihood(y_train, mu_pred, sigma)\n",
+ "# Let's double check we get the right answer before proceeding\n",
+ "print(\"Correct answer = %9.9f, Your answer = %9.9f\"%(0.000010624,likelihood))"
+ ],
+ "metadata": {
+ "id": "1hQxBLoVNlr2"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "You can see that this gives a very small answer, even for this small 1D dataset, and with the model fitting quite well. This is because it is the product of several probabilities, which are all quite small themselves.\n",
+ "This will get out of hand pretty quickly with real datasets -- the likelihood will get so small that we can't represent it with normal finite-precision math\n",
+ "\n",
+ "This is why we use negative log likelihood"
+ ],
+ "metadata": {
+ "id": "HzphKgPfOvlk"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Return the negative log likelihood of the data under the model\n",
+ "def compute_negative_log_likelihood(y_train, mu, sigma):\n",
+ " # TODO -- compute the likelihood of the data -- don't use the likelihood function above -- compute the negative sum of the log probabilities\n",
+ " # Bottom line of equation 5.4 in the notes\n",
+ " # You will need np.sum(), np.log()\n",
+ " # Replace the line below\n",
+ " nll = 0\n",
+ " return nll"
+ ],
+ "metadata": {
+ "id": "dsT0CWiKBmTV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's test this for a homoscedastic (constant sigma) model\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "# Use our neural network to predict the mean of the Gaussian\n",
+ "mu_pred = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+ "# Set the standard deviation to something reasonable\n",
+ "sigma = 0.2\n",
+ "# Compute the log likelihood\n",
+ "nll = compute_negative_log_likelihood(y_train, mu_pred, sigma)\n",
+ "# Let's double check we get the right answer before proceeding\n",
+ "print(\"Correct answer = %9.9f, Your answer = %9.9f\"%(11.452419564,nll))"
+ ],
+ "metadata": {
+ "id": "nVxUXg9rQmwI"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "For good measure, let's compute the sum of squares as well"
+ ],
+ "metadata": {
+ "id": "-S8bXApoWVLG"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Return the squared distance between the predicted \n",
+ "def compute_sum_of_squares(y_train, y_pred):\n",
+ " # TODO -- compute the sum of squared distances between the training data and the model prediction\n",
+ " # Eqn 5.10 in the notes. Make sure that you understand this, and ask questions if you don't\n",
+ " # Replace the line below\n",
+ " sum_of_squares = 0;\n",
+ " return sum_of_squares"
+ ],
+ "metadata": {
+ "id": "I1pjFdHCF4JZ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's test this again\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "# Use our neural network to predict the mean of the Gaussian\n",
+ "y_pred = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+ "# Compute the log likelihood\n",
+ "sum_of_squares = compute_sum_of_squares(y_train, y_pred)\n",
+ "# Let's double check we get the right answer before proceeding\n",
+ "print(\"Correct answer = %9.9f, Your answer = %9.9f\"%(2.020992572,sum_of_squares))"
+ ],
+ "metadata": {
+ "id": "2C40fskIHBx7"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's investigate finding the maximum likelihood / minimum log likelihood / least squares solution. For simplicity, we'll assume that all the parameters are correct except one and look at how the likelihood, log likelihood, and sum of squares change as we manipulate the last parameter. We'll start with overall y offset, beta_1 (formerly phi_0)"
+ ],
+ "metadata": {
+ "id": "OgcRojvPWh4V"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define a range of values for the parameter\n",
+ "beta_1_vals = np.arange(0,1.0,0.01)\n",
+ "# Create some arrays to store the likelihoods, negative log likelihoods and sum of squares\n",
+ "likelihoods = np.zeros_like(beta_1_vals)\n",
+ "nlls = np.zeros_like(beta_1_vals)\n",
+ "sum_squares = np.zeros_like(beta_1_vals)\n",
+ "\n",
+ "# Initialise the parameters\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "sigma = 0.2\n",
+ "for count in range(len(beta_1_vals)):\n",
+ " # Set the value for the parameter\n",
+ " beta_1[0,0] = beta_1_vals[count]\n",
+ " # Run the network with new parameters\n",
+ " mu_pred = y_pred = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+ " # Compute and store the three values\n",
+ " likelihoods[count] = compute_likelihood(y_train, mu_pred, sigma)\n",
+ " nlls[count] = compute_negative_log_likelihood(y_train, mu_pred, sigma)\n",
+ " sum_squares[count] = compute_sum_of_squares(y_train, y_pred)\n",
+ " # Draw the model for every 20th parameter setting\n",
+ " if count % 20 == 0:\n",
+ " # Run the model to get values to plot and plot it.\n",
+ " y_model = shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
+ " plot_univariate_regression(x_model, y_model, x_train, y_train, sigma_model = sigma, title=\"beta1[0]=%3.3f\"%(beta_1[0,0]))\n"
+ ],
+ "metadata": {
+ "id": "pFKtDaAeVU4U"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now let's plot the likelihood, negative log likelihood, and least squares as a function the value of the offset beta1\n",
+ "fig, ax = plt.subplots(1,3)\n",
+ "fig.set_size_inches(10.5, 3.5)\n",
+ "fig.tight_layout(pad=3.0)\n",
+ "ax[0].plot(beta_1_vals, likelihoods); ax[0].set_xlabel('beta_1[0]'); ax[0].set_ylabel('likelihood')\n",
+ "ax[1].plot(beta_1_vals, nlls); ax[1].set_xlabel('beta_1[0]'); ax[1].set_ylabel('negative log likelihood')\n",
+ "ax[2].plot(beta_1_vals, sum_squares); ax[2].set_xlabel('beta_1[0]'); ax[2].set_ylabel('sum of squares')\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "UHXeTa9MagO6"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Hopefully, you can see that the maximum of the likelihood fn is at the same position as the minimum negative log likelihood\n",
+ "# and the least squares solutions\n",
+ "# Let's check that:\n",
+ "print(\"Maximum likelihood = %3.3f, at beta_1=%3.3f\"%( (likelihoods[np.argmax(likelihoods)],beta_1_vals[np.argmax(likelihoods)])))\n",
+ "print(\"Minimum negative log likelihood = %3.3f, at beta_1=%3.3f\"%( (nlls[np.argmin(nlls)],beta_1_vals[np.argmin(nlls)])))\n",
+ "print(\"Least squares = %3.3f, at beta_1=%3.3f\"%( (sum_squares[np.argmin(sum_squares)],beta_1_vals[np.argmin(sum_squares)])))\n",
+ "\n",
+ "# Plot the best model\n",
+ "beta_1[0,0] = beta_1_vals[np.argmin(sum_squares)]\n",
+ "y_model = shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
+ "plot_univariate_regression(x_model, y_model, x_train, y_train, sigma_model = sigma, title=\"beta1=%3.3f\"%(beta_1[0,0]))"
+ ],
+ "metadata": {
+ "id": "aDEPhddNdN4u"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "They all give the same answer. But you can see from the three plots above that the likelihood is very small unless the parameters are almost correct. So in practice, we would work with the negative log likelihood or the least squares."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Loss functions part II\n",
+ "\n",
+ "This practical investigates loss functions. In part I we investigated univariate regression (where the output data $y$ is continuous. Our formulation was based on the normal/Gaussian distribution.\n",
+ "\n",
+ "In this notebook, we investigate binary classification (where the output data is 0 or 1). This will be based on the Bernoulli distribution\n",
+ "\n",
+ "In part III we'll investigate multiclass classification (where the outputs data can take multiple values 1,... K.\n",
+ "\n",
+ "We'll compute loss functions with maximum likelihood and minimum negative log likelihood."
+ ],
+ "metadata": {
+ "id": "jSlFkICHwHQF"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "PYMZ1x-Pv1ht"
+ },
+ "outputs": [],
+ "source": [
+ "# Imports math library\n",
+ "import numpy as np\n",
+ "# Imports plotting library\n",
+ "import matplotlib.pyplot as plt\n",
+ "# Import math Library\n",
+ "import math"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the Rectified Linear Unit (ReLU) function\n",
+ "def ReLU(preactivation):\n",
+ " activation = preactivation.clip(0.0)\n",
+ " return activation\n",
+ "\n",
+ "# Define a shallow neural network\n",
+ "def shallow_nn(x, beta_0, omega_0, beta_1, omaga_1):\n",
+ " # Make sure that input data is (1 x n_data) array\n",
+ " n_data = x.size\n",
+ " x = np.reshape(x,(1,n_data))\n",
+ "\n",
+ " # This runs the network for ALL of the inputs, x at once so we can draw graph\n",
+ " h1 = ReLU(np.matmul(beta_0,np.ones((1,n_data))) + np.matmul(omega_0,x))\n",
+ " model_out = np.matmul(beta_1,np.ones((1,n_data))) + np.matmul(omega_1,h1)\n",
+ " return model_out"
+ ],
+ "metadata": {
+ "id": "Fv7SZR3tv7mV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Utility function for plotting data\n",
+ "def plot_binary_classification(x_model, out_model, lambda_model, x_data = None, y_data = None, title= None):\n",
+ " # Make sure model data are 1D arrays\n",
+ " x_model = np.squeeze(x_model)\n",
+ " out_model = np.squeeze(out_model)\n",
+ " lambda_model = np.squeeze(lambda_model)\n",
+ "\n",
+ " fig, ax = plt.subplots(1,2)\n",
+ " fig.set_size_inches(7.0, 3.5)\n",
+ " fig.tight_layout(pad=3.0)\n",
+ " ax[0].plot(x_model,out_model)\n",
+ " ax[0].set_xlabel('Input, $x$'); ax[0].set_ylabel('Model output')\n",
+ " ax[0].set_xlim([0,1]);ax[0].set_ylim([-4,4])\n",
+ " if title is not None:\n",
+ " ax[0].set_title(title)\n",
+ " ax[1].plot(x_model,lambda_model)\n",
+ " ax[1].set_xlabel('Input, $x$'); ax[1].set_ylabel('$\\lambda$ or Pr(y=1|x)')\n",
+ " ax[1].set_xlim([0,1]);ax[1].set_ylim([-0.05,1.05])\n",
+ " if title is not None:\n",
+ " ax[1].set_title(title)\n",
+ " if x_data is not None:\n",
+ " ax[1].plot(x_data, y_data, 'ko')\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "NRR67ri_1TzN"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Binary classification"
+ ],
+ "metadata": {
+ "id": "PsgLZwsPxauP"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Get parameters for model -- we can call this function to easily reset them\n",
+ "def get_parameters():\n",
+ " # And we'll create a network that approximately fits it\n",
+ " beta_0 = np.zeros((3,1)); # formerly theta_x0\n",
+ " omega_0 = np.zeros((3,1)); # formerly theta_x1\n",
+ " beta_1 = np.zeros((1,1)); # formerly phi_0\n",
+ " omega_1 = np.zeros((1,3)); # formerly phi_x\n",
+ "\n",
+ " beta_0[0,0] = 0.3; beta_0[1,0] = -1.0; beta_0[2,0] = -0.5\n",
+ " omega_0[0,0] = -1.0; omega_0[1,0] = 1.8; omega_0[2,0] = 0.65\n",
+ " beta_1[0,0] = 2.6;\n",
+ " omega_1[0,0] = -24.0; omega_1[0,1] = -8.0; omega_1[0,2] = 50.0\n",
+ "\n",
+ " return beta_0, omega_0, beta_1, omega_1"
+ ],
+ "metadata": {
+ "id": "pUT9Ain_HRim"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Sigmoid function that maps [-infty,infty] to [0,1]\n",
+ "def sigmoid(model_out):\n",
+ " # TODO -- implement the logistic sigmoid function\n",
+ " # Replace this line:\n",
+ " sig_model_out = np.zeros_like(model_out)\n",
+ " return sig_model_out"
+ ],
+ "metadata": {
+ "id": "uFb8h-9IXnIe"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's create some 1D training data\n",
+ "x_train = np.array([0.09291784,0.46809093,0.93089486,0.67612654,0.73441752,0.86847339,\\\n",
+ " 0.49873225,0.51083168,0.18343972,0.99380898,0.27840809,0.38028817,\\\n",
+ " 0.12055708,0.56715537,0.92005746,0.77072270,0.85278176,0.05315950,\\\n",
+ " 0.87168699,0.58858043])\n",
+ "y_train = np.array([0,1,1,0,0,1,\\\n",
+ " 1,0,0,1,0,1,\\\n",
+ " 0,1,1,0,1,0, \\\n",
+ " 1,1])\n",
+ "\n",
+ "# Get parameters for the model\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "\n",
+ "# Define a range of input values\n",
+ "x_model = np.arange(0,1,0.01)\n",
+ "# Run the model to get values to plot and plot it.\n",
+ "model_out= shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
+ "lambda_model = sigmoid(model_out)\n",
+ "plot_binary_classification(x_model, model_out, lambda_model, x_train, y_train)\n"
+ ],
+ "metadata": {
+ "id": "VWzNOt1swFVd"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The left is model output and the right is the model output after the sigmoid has been applied, so it now lies in the range [0,1] and represents the probability, that y=1. The black dots show the training data. We'll compute the the likelihood and the negative log likelihood."
+ ],
+ "metadata": {
+ "id": "MvVX6tl9AEXF"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Return probability under Bernoulli distribution for input x\n",
+ "def bernoulli_distribution(y, lambda_param):\n",
+ " # TODO-- write in the equation for the Bernoulli distribution \n",
+ " # Equation 5.17 from the notes (you will need np.power)\n",
+ " # Replace the line below\n",
+ " prob = np.zeros_like(y)\n",
+ " return prob"
+ ],
+ "metadata": {
+ "id": "YaLdRlEX0FkU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's double check we get the right answer before proceeding\n",
+ "print(\"Correct answer = %3.3f, Your answer = %3.3f\"%(0.8,bernoulli_distribution(0,0.2)))\n",
+ "print(\"Correct answer = %3.3f, Your answer = %3.3f\"%(0.2,bernoulli_distribution(1,0.2)))"
+ ],
+ "metadata": {
+ "id": "4TSL14dqHHbV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's compute the likelihood using this function"
+ ],
+ "metadata": {
+ "id": "R5z_0dzQMF35"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Return the likelihood of all of the data under the model\n",
+ "def compute_likelihood(y_train, lambda_param):\n",
+ " # TODO -- compute the likelihood of the data -- the product of the Bernoulli's probabilities for each data point\n",
+ " # Top line of equation 5.3 in the notes\n",
+ " # You will need np.prod() and the bernoulli_distribution function you used above\n",
+ " # Replace the line below\n",
+ " likelihood = 0\n",
+ " return likelihood"
+ ],
+ "metadata": {
+ "id": "sk4EJSPQ41CK"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's test this \n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "# Use our neural network to predict the mean of the Gaussian\n",
+ "model_out = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+ "lambda_train = sigmoid(model_out)\n",
+ "# Compute the likelihood\n",
+ "likelihood = compute_likelihood(y_train, lambda_train)\n",
+ "# Let's double check we get the right answer before proceeding\n",
+ "print(\"Correct answer = %9.9f, Your answer = %9.9f\"%(0.000070237,likelihood))"
+ ],
+ "metadata": {
+ "id": "1hQxBLoVNlr2"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "You can see that this gives a very small answer, even for this small 1D dataset, and with the model fitting quite well. This is because it is the product of several probabilities, which are all quite small themselves.\n",
+ "This will get out of hand pretty quickly with real datasets -- the likelihood will get so small that we can't represent it with normal finite-precision math\n",
+ "\n",
+ "This is why we use negative log likelihood"
+ ],
+ "metadata": {
+ "id": "HzphKgPfOvlk"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Return the negative log likelihood of the data under the model\n",
+ "def compute_negative_log_likelihood(y_train, lambda_param):\n",
+ " # TODO -- compute the likelihood of the data -- don't use the likelihood function above -- compute the negative sum of the log probabilities\n",
+ " # You will need np.sum(), np.log()\n",
+ " # Replace the line below\n",
+ " nll = 0\n",
+ " return nll"
+ ],
+ "metadata": {
+ "id": "dsT0CWiKBmTV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's test this\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "# Use our neural network to predict the mean of the Gaussian\n",
+ "model_out = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+ "# Set the standard deviation to something reasonable\n",
+ "lambda_train = sigmoid(model_out)\n",
+ "# Compute the log likelihood\n",
+ "nll = compute_negative_log_likelihood(y_train, lambda_train)\n",
+ "# Let's double check we get the right answer before proceeding\n",
+ "print(\"Correct answer = %9.9f, Your answer = %9.9f\"%(9.563639387,nll))"
+ ],
+ "metadata": {
+ "id": "nVxUXg9rQmwI"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's investigate finding the maximum likelihood / minimum log likelihood / least squares solution. For simplicity, we'll assume that all the parameters are fixed except one and look at how the likelihood and log likelihood change as we manipulate the last parameter. We'll start with overall y_offset, beta_1 (formerly phi_0)"
+ ],
+ "metadata": {
+ "id": "OgcRojvPWh4V"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Return the likelihood of all of the data under the model\n",
+ "def compute_likelihood(y_train, lambda_param):\n",
+ " # TODO -- compute the likelihood of the data -- the product of the Bernoulli probabilities for each data point\n",
+ " # Top line of equation 5.3 in the notes\n",
+ " # You will need np.prod() and the bernoulli_distribution function you used above\n",
+ " # Replace the line below\n",
+ " likelihood = 0\n",
+ " return likelihood"
+ ],
+ "metadata": {
+ "id": "zpS7o6liCx7f"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define a range of values for the parameter\n",
+ "beta_1_vals = np.arange(-2,6.0,0.1)\n",
+ "# Create some arrays to store the likelihoods, negative log likelihoods\n",
+ "likelihoods = np.zeros_like(beta_1_vals)\n",
+ "nlls = np.zeros_like(beta_1_vals)\n",
+ "\n",
+ "# Initialise the parameters\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "for count in range(len(beta_1_vals)):\n",
+ " # Set the value for the parameter\n",
+ " beta_1[0,0] = beta_1_vals[count]\n",
+ " # Run the network with new parameters\n",
+ " model_out = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+ " lambda_train = sigmoid(model_out)\n",
+ " # Compute and store the three values\n",
+ " likelihoods[count] = compute_likelihood(y_train,lambda_train)\n",
+ " nlls[count] = compute_negative_log_likelihood(y_train, lambda_train)\n",
+ " # Draw the model for every 20th parameter setting\n",
+ " if count % 20 == 0:\n",
+ " # Run the model to get values to plot and plot it.\n",
+ " model_out = shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
+ " lambda_model = sigmoid(model_out)\n",
+ " plot_binary_classification(x_model, model_out, lambda_model, x_train, y_train, title=\"beta_1[0]=%3.3f\"%(beta_1[0,0]))\n"
+ ],
+ "metadata": {
+ "id": "pFKtDaAeVU4U"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now let's plot the likelihood, negative log likelihood, and least squares as a function the value of the offset beta1\n",
+ "fig, ax = plt.subplots(1,2)\n",
+ "fig.set_size_inches(10.5, 3.5)\n",
+ "fig.tight_layout(pad=3.0)\n",
+ "ax[0].plot(beta_1_vals, likelihoods); ax[0].set_xlabel('beta_1[0]'); ax[0].set_ylabel('likelihood')\n",
+ "ax[1].plot(beta_1_vals, nlls); ax[1].set_xlabel('beta_1[0]'); ax[1].set_ylabel('negative log likelihood')\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "UHXeTa9MagO6"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Hopefully, you can see that the maximum of the likelihood fn is at the same position as the minimum negative log likelihood\n",
+ "# and the least squares solutions\n",
+ "# Let's check that:\n",
+ "print(\"Maximum likelihood = %f, at beta_1=%3.3f\"%( (likelihoods[np.argmax(likelihoods)],beta_1_vals[np.argmax(likelihoods)])))\n",
+ "print(\"Minimum negative log likelihood = %f, at beta_1=%3.3f\"%( (nlls[np.argmin(nlls)],beta_1_vals[np.argmin(nlls)])))\n",
+ "\n",
+ "# Plot the best model\n",
+ "beta_1[0,0] = beta_1_vals[np.argmin(nlls)]\n",
+ "model_out = shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
+ "lambda_model = sigmoid(model_out)\n",
+ "plot_binary_classification(x_model, model_out, lambda_model, x_train, y_train, title=\"beta_1[0]=%3.3f\"%(beta_1[0,0]))\n"
+ ],
+ "metadata": {
+ "id": "aDEPhddNdN4u"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "They both give the same answer. But you can see from the likelihood above that the likelihood is very small unless the parameters are almost correct. So in practice, we would work with the negative log likelihood."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Loss functions part III\n",
+ "\n",
+ "This practical investigates loss functions. In part I we investigated univariate regression (where the output data $y$ is continuous. Our formulation was based on the normal/Gaussian distribution.\n",
+ "In part II we investigated binary classification (where the output data is 0 or 1). This will be based on the Bernoulli distribution."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **CM20315 Shallow neural networks**\n",
+ "\n",
+ "The purpose of this practical is to gain some familiarity with shallow neural networks. It explores using different numbers of inputs and outputs, hidden units and activation functions. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Training\n",
+ "\n",
+ "We now have a model and a loss function which we can use to judge how good that model is. It's time to put the \"learning\" into machine learning.\n",
+ "\n",
+ "Learning involves finding the parameters that minimize the loss. That might seems like it's not too hard, but modern models might have billions of parameters. There's an exponential number of possible parameter combinations, and there's no way we can make any progress with exhaustive search.\n",
+ "\n",
+ "We'll build this up in stages. In this practical, we'll just consider 1D search using a bracketing approach. In part II, we'll extend to fitting the linear regression model (which has a convex loss function). Then in part III, we'll consider non-convex loss functions\n"
+ ],
+ "metadata": {
+ "id": "el8l05WQEO46"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "xhmIOLiZELV_"
+ },
+ "outputs": [],
+ "source": [
+ "# import libraries\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's create a simple 1D function\n",
+ "def loss_function(phi):\n",
+ " return 1- 0.5 * np.exp(-(phi-0.65)*(phi-0.65)/0.1) - 0.45 *np.exp(-(phi-0.35)*(phi-0.35)/0.02)\n",
+ "\n",
+ "def draw_function(loss_function,a=None, b=None, c=None, d=None):\n",
+ " # Plot the function \n",
+ " phi_plot = np.arange(0,1,0.01);\n",
+ " fig,ax = plt.subplots()\n",
+ " ax.plot(phi_plot,loss_function(phi_plot),'r-')\n",
+ " ax.set_xlim(0,1); ax.set_ylim(0,1)\n",
+ " ax.set_xlabel('$\\phi$'); ax.set_ylabel('$L[\\phi]$')\n",
+ " if a is not None and b is not None and c is not None and d is not None:\n",
+ " plt.axvspan(a, d, facecolor='k', alpha=0.2)\n",
+ " ax.plot([a,a],[0,1],'b-')\n",
+ " ax.plot([b,b],[0,1],'b-')\n",
+ " ax.plot([c,c],[0,1],'b-')\n",
+ " ax.plot([d,d],[0,1],'b-')\n",
+ " plt.show()\n"
+ ],
+ "metadata": {
+ "id": "qFRe9POHF2le"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Draw this function\n",
+ "draw_function(loss_function)"
+ ],
+ "metadata": {
+ "id": "TXx1Tpd1Tl-I"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now lets create a line search procedure to find the minimum in the range 0,1"
+ ],
+ "metadata": {
+ "id": "QU5mdGvpTtEG"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def line_search(loss_function, thresh=.0001, max_iter = 10, draw_flag = False):\n",
+ "\n",
+ " # Initialize four points along the range we are going to search\n",
+ " a = 0\n",
+ " b = 0.33\n",
+ " c = 0.66\n",
+ " d = 1.0\n",
+ " n_iter =0;\n",
+ " \n",
+ " # While we haven't found the minimum closely enough\n",
+ " while np.abs(b-c) > thresh and n_iter < max_iter:\n",
+ " # Increment iteration counter (just to prevent an infinite loop)\n",
+ " n_iter = n_iter+1\n",
+ " # Calculate all four points\n",
+ " lossa = loss_function(a)\n",
+ " lossb = loss_function(b)\n",
+ " lossc = loss_function(c)\n",
+ " lossd = loss_function(d)\n",
+ "\n",
+ " if draw_flag:\n",
+ " draw_function(loss_function, a,b,c,d)\n",
+ "\n",
+ " print('Iter %d, a=%3.3f, b=%3.3f, c=%3.3f, d=%3.3f'%(n_iter, a,b,c,d))\n",
+ "\n",
+ " # Rule #1 If point A is less than points B, C, and D then halve values of B,C, and D\n",
+ " # i.e. bring them closer to the original point\n",
+ " # TODO REPLACE THE BLOCK OF CODE BELOW WITH THIS RULE\n",
+ " if (0):\n",
+ " continue;\n",
+ "\n",
+ " # Rule #2 If point b is less than point c then\n",
+ " # then point d becomes point c, and\n",
+ " # point b becomes 1/3 between a and new d\n",
+ " # point c becomes 2/3 between a and new d \n",
+ " # TODO REPLACE THE BLOCK OF CODE BELOW WITH THIS RULE\n",
+ " if (0):\n",
+ " continue;\n",
+ "\n",
+ " # Rule #3 If point c is less than point b then\n",
+ " # then point a becomes point b, and\n",
+ " # point b becomes 1/3 between new a and d\n",
+ " # point c becomes 2/3 between new a and d \n",
+ " # TODO REPLACE THE BLOCK OF CODE BELOW WITH THIS RULE\n",
+ " if(0):\n",
+ " continue\n",
+ "\n",
+ "\n",
+ " # TODO -- FINAL SOLUTION IS AVERAGE OF B and C\n",
+ " # REPLACE THIS LINE\n",
+ " soln = 1\n",
+ " \n",
+ " return soln"
+ ],
+ "metadata": {
+ "id": "K-NTHpAAHlCl"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "soln = line_search(loss_function, draw_flag=True)\n",
+ "print('Soln = %3.3f, loss = %3.3f'%(soln,loss_function(soln)))"
+ ],
+ "metadata": {
+ "id": "YVq6rmaWRD2M"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "tOLd0gtdRLLS"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
diff --git a/CM20315/CM20315_Training_II.ipynb b/CM20315/CM20315_Training_II.ipynb
new file mode 100644
index 0000000..ac09b14
--- /dev/null
+++ b/CM20315/CM20315_Training_II.ipynb
@@ -0,0 +1,427 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "collapsed_sections": [],
+ "authorship_tag": "ABX9TyMwBvBF5E8ERRBCqN9x6Dp5",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Training II\n",
+ "\n",
+ "We now have a model and a loss function which we can use to judge how good that model is. It's time to put the \"learning\" into machine learning.\n",
+ "\n",
+ "Learning involves finding the parameters that minimize the loss. That might seems like it's not too hard, but modern models might have billions of parameters. There's an exponential number of possible parameter combinations, and there's no way we can make any progress with exhaustive search.\n",
+ "\n",
+ "In part I we considered 1D search using a bracketing approach. In this part, we'll extend to fitting the linear regression model (which has a convex loss function). Then in part III, we'll consider a non-convex loss function and implement stochastic gradient descent.\n",
+ "\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "el8l05WQEO46"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "xhmIOLiZELV_"
+ },
+ "outputs": [],
+ "source": [
+ "# import libraries\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from matplotlib import cm\n",
+ "from matplotlib.colors import ListedColormap"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's create our training data 12 pairs {x_i, y_i}\n",
+ "# We'll try to fit the straight line model to these data\n",
+ "data = np.array([[0.03,0.19,0.34,0.46,0.78,0.81,1.08,1.18,1.39,1.60,1.65,1.90],\n",
+ " [0.67,0.85,1.05,1.00,1.40,1.50,1.30,1.54,1.55,1.68,1.73,1.60]])"
+ ],
+ "metadata": {
+ "id": "4cRkrh9MZ58Z"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's define our model\n",
+ "def model(phi,x):\n",
+ " y_pred = phi[0]+phi[1] * x\n",
+ " return y_pred"
+ ],
+ "metadata": {
+ "id": "WQUERmb2erAe"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Draw model\n",
+ "def draw_model(data,model,phi,title=None):\n",
+ " x_model = np.arange(0,2,0.01)\n",
+ " y_model = model(phi,x_model)\n",
+ "\n",
+ " fix, ax = plt.subplots()\n",
+ " ax.plot(data[0,:],data[1,:],'bo')\n",
+ " ax.plot(x_model,y_model,'m-')\n",
+ " ax.set_xlim([0,2]);ax.set_ylim([0,2])\n",
+ " ax.set_xlabel('x'); ax.set_ylabel('y')\n",
+ " ax.set_aspect('equal')\n",
+ " if title is not None:\n",
+ " ax.set_title(title)\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "qFRe9POHF2le"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Initialize the parameters and draw the model\n",
+ "phi = np.zeros((2,1))\n",
+ "phi[0] = 0.6 # Intercept\n",
+ "phi[1] = -0.2 # Slope\n",
+ "draw_model(data,model,phi, \"Initial parameters\")\n"
+ ],
+ "metadata": {
+ "id": "TXx1Tpd1Tl-I"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now lets create compute the sum of squares loss for the training data"
+ ],
+ "metadata": {
+ "id": "QU5mdGvpTtEG"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def compute_loss(data_x, data_y, model, phi):\n",
+ " # TODO -- Write this function -- replace the line below\n",
+ " # First make model predictions from data x\n",
+ " # Then compute the squared difference between the predictions and true y values\n",
+ " # Then sum them all and return\n",
+ " loss = 0\n",
+ " return loss"
+ ],
+ "metadata": {
+ "id": "I7dqTY2Gg7CR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's just test that we got that right"
+ ],
+ "metadata": {
+ "id": "eB5DQvU5hYNx"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "loss = compute_loss(data[0,:],data[1,:],model,np.array([[0.6],[-0.2]]))\n",
+ "print('Your loss = %3.3f, Correct loss = %3.3f'%(loss, 12.367))"
+ ],
+ "metadata": {
+ "id": "Ty05UtEEg9tc"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's plot the whole loss function"
+ ],
+ "metadata": {
+ "id": "F3trnavPiHpH"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def draw_loss_function(compute_loss, data, model, phi_iters = None):\n",
+ " # Define pretty colormap\n",
+ " my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ " my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ " r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ " g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ " b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ " my_colormap = ListedColormap(np.vstack((r,g,b)).transpose()/255.0)\n",
+ "\n",
+ " # Make grid of intercept/slope values to plot\n",
+ " intercepts_mesh, slopes_mesh = np.meshgrid(np.arange(0.0,2.0,0.02), np.arange(-1.0,1.0,0.002))\n",
+ " loss_mesh = np.zeros_like(slopes_mesh)\n",
+ " # Compute loss for every set of parameters\n",
+ " for idslope, slope in np.ndenumerate(slopes_mesh):\n",
+ " loss_mesh[idslope] = compute_loss(data[0,:], data[1,:], model, np.array([[intercepts_mesh[idslope]], [slope]]))\n",
+ "\n",
+ " fig,ax = plt.subplots()\n",
+ " fig.set_size_inches(8,8)\n",
+ " ax.contourf(intercepts_mesh,slopes_mesh,loss_mesh,256,cmap=my_colormap)\n",
+ " ax.contour(intercepts_mesh,slopes_mesh,loss_mesh,40,colors=['#80808080'])\n",
+ " if phi_iters is not None:\n",
+ " ax.plot(phi_iters[0,:], phi_iters[1,:],'go-')\n",
+ " ax.set_ylim([1,-1])\n",
+ " ax.set_xlabel('Intercept $\\phi_{0}$'); ax.set_ylabel('Slope, $\\phi_{1}$')\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "K-NTHpAAHlCl"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "draw_loss_function(compute_loss, data, model)"
+ ],
+ "metadata": {
+ "id": "l8HbvIupnTME"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's compute the gradient vector for a given set of parameters:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\frac{\\partial L}{\\partial \\boldsymbol\\phi} = \\begin{bmatrix}\\frac{\\partial L}{\\partial \\phi_0} \\\\\\frac{\\partial L}{\\partial \\phi_1} \\end{bmatrix}.\n",
+ "\\end{equation}"
+ ],
+ "metadata": {
+ "id": "s9Duf05WqqSC"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# These are in the lecture slides and notes, but worth trying to calculate them yourself to \n",
+ "# check that you get them right. Write out the expression for the sum of squares loss and take the\n",
+ "# derivative with respect to phi0 and phi1\n",
+ "def compute_gradient(data_x, data_y, phi):\n",
+ " # TODO -- write this function, replacing the lines below\n",
+ " dl_dphi0 = 0.0\n",
+ " dl_dphi1 = 0.0\n",
+ "\n",
+ " # Return the gradient\n",
+ " return np.array([[dl_dphi0],[dl_dphi1]])"
+ ],
+ "metadata": {
+ "id": "UpswmkL2qwBT"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "We can check we got this right using a trick known as **finite differences**. If we evaluate the function and then change one of the parameters by a very small amount and normalize by that amount, we get an approximation to the gradient, so:\n",
+ "\n",
+ "\\begin{eqnarray}\n",
+ "\\frac{\\partial L}{\\partial \\phi_{0}}&\\approx & \\frac{L[\\phi_0+\\delta, \\phi_1]-L[\\phi_0, \\phi_1]}{\\delta}\\\\\n",
+ "\\frac{\\partial L}{\\partial \\phi_{1}}&\\approx & \\frac{L[\\phi_0, \\phi_1+\\delta]-L[\\phi_0, \\phi_1]}{\\delta}\n",
+ "\\end{eqnarray}\n",
+ "\n",
+ "We don't do this when there are many parameters; for a million parameters, we would have to evaluate the loss function two million times, and usually computing the gradients directly is much more efficient."
+ ],
+ "metadata": {
+ "id": "RS1nEcYVuEAM"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Compute the gradient using your function\n",
+ "gradient = compute_gradient(data[0,:],data[1,:], phi)\n",
+ "print(\"Your gradients: (%3.3f,%3.3f)\"%(gradient[0],gradient[1]))\n",
+ "# Approximate the gradients with finite differences\n",
+ "delta = 0.0001\n",
+ "dl_dphi0_est = (compute_loss(data[0,:],data[1,:],model,phi+np.array([[delta],[0]])) - \\\n",
+ " compute_loss(data[0,:],data[1,:],model,phi))/delta\n",
+ "dl_dphi1_est = (compute_loss(data[0,:],data[1,:],model,phi+np.array([[0],[delta]])) - \\\n",
+ " compute_loss(data[0,:],data[1,:],model,phi))/delta\n",
+ "print(\"Approx gradients: (%3.3f,%3.3f)\"%(dl_dphi0_est,dl_dphi1_est))\n"
+ ],
+ "metadata": {
+ "id": "QuwAHN7yt-gi"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now we are ready to perform gradient descent. We'll need to use our line search routine from part I, which I've reproduced here plus the helper function loss_function_1D that converts from a 2D problem to a 1D problem"
+ ],
+ "metadata": {
+ "id": "5EIjMM9Fw2eT"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def loss_function_1D(dist_prop, data, model, phi_start, gradient):\n",
+ " # Return the loss after moving this far\n",
+ " return compute_loss(data[0,:], data[1,:], model, phi_start+ gradient * dist_prop)\n",
+ "\n",
+ "def line_search(data, model, phi, gradient, thresh=.00001, max_dist = 0.1, max_iter = 15, verbose=False):\n",
+ " # Initialize four points along the range we are going to search\n",
+ " a = 0\n",
+ " b = 0.33 * max_dist\n",
+ " c = 0.66 * max_dist\n",
+ " d = 1.0 * max_dist\n",
+ " n_iter =0;\n",
+ " \n",
+ " # While we haven't found the minimum closely enough\n",
+ " while np.abs(b-c) > thresh and n_iter < max_iter:\n",
+ " # Increment iteration counter (just to prevent an infinite loop)\n",
+ " n_iter = n_iter+1\n",
+ " # Calculate all four points\n",
+ " lossa = loss_function_1D(a, data, model, phi,gradient)\n",
+ " lossb = loss_function_1D(b, data, model, phi,gradient)\n",
+ " lossc = loss_function_1D(c, data, model, phi,gradient)\n",
+ " lossd = loss_function_1D(d, data, model, phi,gradient)\n",
+ "\n",
+ " if verbose:\n",
+ " print('Iter %d, a=%3.3f, b=%3.3f, c=%3.3f, d=%3.3f'%(n_iter, a,b,c,d))\n",
+ " print('a %f, b%f, c%f, d%f'%(lossa,lossb,lossc,lossd))\n",
+ "\n",
+ " # Rule #1 If point A is less than points B, C, and D then halve points B,C, and D\n",
+ " if np.argmin((lossa,lossb,lossc,lossd))==0:\n",
+ " b = b/2\n",
+ " c = c/2\n",
+ " d = d/2\n",
+ " continue;\n",
+ "\n",
+ " # Rule #2 If point b is less than point c then\n",
+ " # then point d becomes point c, and\n",
+ " # point b becomes 1/3 between a and new d\n",
+ " # point c becomes 2/3 between a and new d \n",
+ " if lossb < lossc:\n",
+ " d = c\n",
+ " b = a+ (d-a)/3\n",
+ " c = a+ 2*(d-a)/3\n",
+ " continue\n",
+ "\n",
+ " # Rule #2 If point c is less than point b then\n",
+ " # then point a becomes point b, and\n",
+ " # point b becomes 1/3 between new a and d\n",
+ " # point c becomes 2/3 between new a and d \n",
+ " a = b\n",
+ " b = a+ (d-a)/3\n",
+ " c = a+ 2*(d-a)/3\n",
+ " \n",
+ " # Return average of two middle points\n",
+ " return (b+c)/2.0"
+ ],
+ "metadata": {
+ "id": "XrJ2gQjfw1XP"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def gradient_descent_step(phi, data, model):\n",
+ " # TODO -- update Phi with the gradient descent step\n",
+ " # 1. Compute the gradient\n",
+ " # 2. Find the best step size alpha using line search (use minus 1 times the gradient so it searches in the right direction)\n",
+ " # 3. Update the parameters phi \n",
+ "\n",
+ " return phi"
+ ],
+ "metadata": {
+ "id": "YVq6rmaWRD2M"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Initialize the parameters and draw the model\n",
+ "n_steps = 10\n",
+ "phi_all = np.zeros((2,n_steps+1))\n",
+ "phi_all[0,0] = 1.6\n",
+ "phi_all[1,0] = -0.5\n",
+ "\n",
+ "# Measure loss and draw initial model\n",
+ "loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,0:1])\n",
+ "draw_model(data,model,phi_all[:,0:1], \"Initial parameters, Loss = %f\"%(loss))\n",
+ "\n",
+ "for c_step in range (n_steps):\n",
+ " # Do gradient descent step\n",
+ " phi_all[:,c_step+1:c_step+2] = gradient_descent_step(phi_all[:,c_step:c_step+1],data, model)\n",
+ " # Measure loss and draw model\n",
+ " loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,c_step+1:c_step+2])\n",
+ " draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
+ "\n",
+ "draw_loss_function(compute_loss, data, model,phi_all)\n"
+ ],
+ "metadata": {
+ "id": "tOLd0gtdRLLS"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "Oi8ZlH0ptLqA"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
diff --git a/CM20315/CM20315_Training_III.ipynb b/CM20315/CM20315_Training_III.ipynb
new file mode 100644
index 0000000..f91c021
--- /dev/null
+++ b/CM20315/CM20315_Training_III.ipynb
@@ -0,0 +1,585 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "collapsed_sections": [],
+ "authorship_tag": "ABX9TyMzgGVp+/BUCXimg7Ip9lhp",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Training III\n",
+ "\n",
+ "We now have a model and a loss function which we can use to judge how good that model is. It's time to put the \"learning\" into machine learning.\n",
+ "\n",
+ "Learning involves finding the parameters that minimize the loss. That might seems like it's not too hard, but modern models might have billions of parameters. There's an exponential number of possible parameter combinations, and there's no way we can make any progress with exhaustive search.\n",
+ "\n",
+ "In part I we considered 1D search using a bracketing approach. In part II we experimented with fitting a linear regression model (which has a convex loss function). In this part, we'll fit the Gabor model, which has a non-convex loss function.\n",
+ "\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "el8l05WQEO46"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "xhmIOLiZELV_"
+ },
+ "outputs": [],
+ "source": [
+ "# import libraries\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from matplotlib import cm\n",
+ "from matplotlib.colors import ListedColormap"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's create our training data 30 pairs {x_i, y_i}\n",
+ "# We'll try to fit the Gabor model to these data\n",
+ "data = np.array([[-1.920e+00,-1.422e+01,1.490e+00,-1.940e+00,-2.389e+00,-5.090e+00,\n",
+ " -8.861e+00,3.578e+00,-6.010e+00,-6.995e+00,3.634e+00,8.743e-01,\n",
+ " -1.096e+01,4.073e-01,-9.467e+00,8.560e+00,1.062e+01,-1.729e-01,\n",
+ " 1.040e+01,-1.261e+01,1.574e-01,-1.304e+01,-2.156e+00,-1.210e+01,\n",
+ " -1.119e+01,2.902e+00,-8.220e+00,-1.179e+01,-8.391e+00,-4.505e+00],\n",
+ " [-1.051e+00,-2.482e-02,8.896e-01,-4.943e-01,-9.371e-01,4.306e-01,\n",
+ " 9.577e-03,-7.944e-02 ,1.624e-01,-2.682e-01,-3.129e-01,8.303e-01,\n",
+ " -2.365e-02,5.098e-01,-2.777e-01,3.367e-01,1.927e-01,-2.222e-01,\n",
+ " 6.352e-02,6.888e-03,3.224e-02,1.091e-02,-5.706e-01,-5.258e-02,\n",
+ " -3.666e-02,1.709e-01,-4.805e-02,2.008e-01,-1.904e-01,5.952e-01]])"
+ ],
+ "metadata": {
+ "id": "4cRkrh9MZ58Z"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's define our model\n",
+ "def model(phi,x):\n",
+ " sin_component = np.sin(phi[0] + 0.06 * phi[1] * x)\n",
+ " gauss_component = np.exp(-(phi[0] + 0.06 * phi[1] * x) * (phi[0] + 0.06 * phi[1] * x) / 32)\n",
+ " y_pred= sin_component * gauss_component\n",
+ " return y_pred"
+ ],
+ "metadata": {
+ "id": "WQUERmb2erAe"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Draw model\n",
+ "def draw_model(data,model,phi,title=None):\n",
+ " x_model = np.arange(-15,15,0.1)\n",
+ " y_model = model(phi,x_model)\n",
+ "\n",
+ " fix, ax = plt.subplots()\n",
+ " ax.plot(data[0,:],data[1,:],'bo')\n",
+ " ax.plot(x_model,y_model,'m-')\n",
+ " ax.set_xlim([-15,15]);ax.set_ylim([-1,1])\n",
+ " ax.set_xlabel('x'); ax.set_ylabel('y')\n",
+ " if title is not None:\n",
+ " ax.set_title(title)\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "qFRe9POHF2le"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Initialize the parmaeters and draw the model\n",
+ "phi = np.zeros((2,1))\n",
+ "phi[0] = -5 # Horizontal offset\n",
+ "phi[1] = 25 # Frequency\n",
+ "draw_model(data,model,phi, \"Initial parameters\")\n"
+ ],
+ "metadata": {
+ "id": "TXx1Tpd1Tl-I"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now lets create compute the sum of squares loss for the training data"
+ ],
+ "metadata": {
+ "id": "QU5mdGvpTtEG"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def compute_loss(data_x, data_y, model, phi):\n",
+ " # TODO -- Write this function -- replace the line below\n",
+ " # TODO -- First make model predictions from data x\n",
+ " # TODO -- Then compute the squared difference between the predictions and true y values\n",
+ " # TODO -- Then sum them all and return\n",
+ " loss = 0\n",
+ " return loss"
+ ],
+ "metadata": {
+ "id": "I7dqTY2Gg7CR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's just test that we got that right"
+ ],
+ "metadata": {
+ "id": "eB5DQvU5hYNx"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "loss = compute_loss(data[0,:],data[1,:],model,np.array([[0.6],[-0.2]]))\n",
+ "print('Your loss = %3.3f, Correct loss = %3.3f'%(loss, 16.419))"
+ ],
+ "metadata": {
+ "id": "Ty05UtEEg9tc"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's plot the whole loss function"
+ ],
+ "metadata": {
+ "id": "F3trnavPiHpH"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def draw_loss_function(compute_loss, data, model, phi_iters = None):\n",
+ " # Define pretty colormap\n",
+ " my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ " my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ " r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ " g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ " b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ " my_colormap = ListedColormap(np.vstack((r,g,b)).transpose()/255.0)\n",
+ "\n",
+ " # Make grid of intercept/slope values to plot\n",
+ " offsets_mesh, freqs_mesh = np.meshgrid(np.arange(-10,10.0,0.1), np.arange(2.5,22.5,0.1))\n",
+ " loss_mesh = np.zeros_like(freqs_mesh)\n",
+ " # Compute loss for every set of parameters\n",
+ " for idslope, slope in np.ndenumerate(freqs_mesh):\n",
+ " loss_mesh[idslope] = compute_loss(data[0,:], data[1,:], model, np.array([[offsets_mesh[idslope]], [slope]]))\n",
+ "\n",
+ " fig,ax = plt.subplots()\n",
+ " fig.set_size_inches(8,8)\n",
+ " ax.contourf(offsets_mesh,freqs_mesh,loss_mesh,256,cmap=my_colormap)\n",
+ " ax.contour(offsets_mesh,freqs_mesh,loss_mesh,20,colors=['#80808080'])\n",
+ " if phi_iters is not None:\n",
+ " ax.plot(phi_iters[0,:], phi_iters[1,:],'go-')\n",
+ " ax.set_ylim([2.5,22.5])\n",
+ " ax.set_xlabel('Offset $\\phi_{0}$'); ax.set_ylabel('Frequency, $\\phi_{1}$')\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "K-NTHpAAHlCl"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "draw_loss_function(compute_loss, data, model)"
+ ],
+ "metadata": {
+ "id": "l8HbvIupnTME"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's compute the gradient vector for a given set of parameters:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\frac{\\partial L}{\\partial \\boldsymbol\\phi} = \\begin{bmatrix}\\frac{\\partial L}{\\partial \\phi_0} \\\\\\frac{\\partial L}{\\partial \\phi_1} \\end{bmatrix}.\n",
+ "\\end{equation}"
+ ],
+ "metadata": {
+ "id": "s9Duf05WqqSC"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# These came from writing out the expression for the sum of squares loss and taking the\n",
+ "# derivative with respect to phi0 and phi1. It was a lot of hassle to get it right!\n",
+ "def gabor_deriv_phi0(data_x,data_y,phi0, phi1):\n",
+ " x = 0.06 * phi1 * data_x + phi0\n",
+ " y = data_y \n",
+ " cos_component = np.cos(x)\n",
+ " sin_component = np.sin(x)\n",
+ " gauss_component = np.exp(-0.5 * x *x / 16)\n",
+ " deriv = cos_component * gauss_component - sin_component * gauss_component * x / 16\n",
+ " deriv = 2* deriv * (sin_component * gauss_component - y)\n",
+ " return np.sum(deriv)\n",
+ "\n",
+ "def gabor_deriv_phi1(data_x, data_y,phi0, phi1):\n",
+ " x = 0.06 * phi1 * data_x + phi0\n",
+ " y = data_y \n",
+ " cos_component = np.cos(x)\n",
+ " sin_component = np.sin(x)\n",
+ " gauss_component = np.exp(-0.5 * x *x / 16)\n",
+ " deriv = 0.06 * data_x * cos_component * gauss_component - 0.06 * data_x*sin_component * gauss_component * x / 16\n",
+ " deriv = 2*deriv * (sin_component * gauss_component - y)\n",
+ " return np.sum(deriv)\n",
+ "\n",
+ "def compute_gradient(data_x, data_y, phi):\n",
+ " dl_dphi0 = gabor_deriv_phi0(data_x, data_y, phi[0],phi[1])\n",
+ " dl_dphi1 = gabor_deriv_phi1(data_x, data_y, phi[0],phi[1])\n",
+ " # Return the gradient\n",
+ " return np.array([[dl_dphi0],[dl_dphi1]])"
+ ],
+ "metadata": {
+ "id": "UpswmkL2qwBT"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "We can check we got this right using a trick known as **finite differences**. If we evaluate the function and then change one of the parameters by a very small amount and normalize by that amount, we get an approximation to the gradient, so:\n",
+ "\n",
+ "\\begin{eqnarray}\n",
+ "\\frac{\\partial L}{\\partial \\phi_{0}}&\\approx & \\frac{L[\\phi_0+\\delta, \\phi_1]-L[\\phi_0, \\phi_1]}{\\delta}\\\\\n",
+ "\\frac{\\partial L}{\\partial \\phi_{1}}&\\approx & \\frac{L[\\phi_0, \\phi_1+\\delta]-L[\\phi_0, \\phi_1]}{\\delta}\n",
+ "\\end{eqnarray}\n",
+ "\n",
+ "We don't do this when there are many parameters; for a million parameters, we would have to evaluate the loss function two million times, and usually computing the gradients directly is much more efficient."
+ ],
+ "metadata": {
+ "id": "RS1nEcYVuEAM"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Compute the gradient using your function\n",
+ "gradient = compute_gradient(data[0,:],data[1,:], phi)\n",
+ "print(\"Your gradients: (%3.3f,%3.3f)\"%(gradient[0],gradient[1]))\n",
+ "# Approximate the gradients with finite differences\n",
+ "delta = 0.0001\n",
+ "dl_dphi0_est = (compute_loss(data[0,:],data[1,:],model,phi+np.array([[delta],[0]])) - \\\n",
+ " compute_loss(data[0,:],data[1,:],model,phi))/delta\n",
+ "dl_dphi1_est = (compute_loss(data[0,:],data[1,:],model,phi+np.array([[0],[delta]])) - \\\n",
+ " compute_loss(data[0,:],data[1,:],model,phi))/delta\n",
+ "print(\"Approx gradients: (%3.3f,%3.3f)\"%(dl_dphi0_est,dl_dphi1_est))\n"
+ ],
+ "metadata": {
+ "id": "QuwAHN7yt-gi"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now we are ready to perform gradient descent. We'll need to use our line search routine from part I, which I've reproduced here plus the helper function loss_function_1D that converts from a 2D problem to a 1D problem"
+ ],
+ "metadata": {
+ "id": "5EIjMM9Fw2eT"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def loss_function_1D(dist_prop, data, model, phi_start, gradient):\n",
+ " # Return the loss after moving this far\n",
+ " return compute_loss(data[0,:], data[1,:], model, phi_start+ gradient * dist_prop)\n",
+ "\n",
+ "def line_search(data, model, phi, gradient, thresh=.00001, max_dist = 0.1, max_iter = 15, verbose=False):\n",
+ " # Initialize four points along the range we are going to search\n",
+ " a = 0\n",
+ " b = 0.33 * max_dist\n",
+ " c = 0.66 * max_dist\n",
+ " d = 1.0 * max_dist\n",
+ " n_iter =0;\n",
+ " \n",
+ " # While we haven't found the minimum closely enough\n",
+ " while np.abs(b-c) > thresh and n_iter < max_iter:\n",
+ " # Increment iteration counter (just to prevent an infinite loop)\n",
+ " n_iter = n_iter+1\n",
+ " # Calculate all four points\n",
+ " lossa = loss_function_1D(a, data, model, phi,gradient)\n",
+ " lossb = loss_function_1D(b, data, model, phi,gradient)\n",
+ " lossc = loss_function_1D(c, data, model, phi,gradient)\n",
+ " lossd = loss_function_1D(d, data, model, phi,gradient)\n",
+ "\n",
+ " if verbose:\n",
+ " print('Iter %d, a=%3.3f, b=%3.3f, c=%3.3f, d=%3.3f'%(n_iter, a,b,c,d))\n",
+ " print('a %f, b%f, c%f, d%f'%(lossa,lossb,lossc,lossd))\n",
+ "\n",
+ " # Rule #1 If point A is less than points B, C, and D then halve points B,C, and D\n",
+ " if np.argmin((lossa,lossb,lossc,lossd))==0:\n",
+ " b = b/2\n",
+ " c = c/2\n",
+ " d = d/2\n",
+ " continue;\n",
+ "\n",
+ " # Rule #2 If point b is less than point c then\n",
+ " # then point d becomes point c, and\n",
+ " # point b becomes 1/3 between a and new d\n",
+ " # point c becomes 2/3 between a and new d \n",
+ " if lossb < lossc:\n",
+ " d = c\n",
+ " b = a+ (d-a)/3\n",
+ " c = a+ 2*(d-a)/3\n",
+ " continue\n",
+ "\n",
+ " # Rule #2 If point c is less than point b then\n",
+ " # then point a becomes point b, and\n",
+ " # point b becomes 1/3 between new a and d\n",
+ " # point c becomes 2/3 between new a and d \n",
+ " a = b\n",
+ " b = a+ (d-a)/3\n",
+ " c = a+ 2*(d-a)/3\n",
+ " \n",
+ " # Return average of two middle points\n",
+ " return (b+c)/2.0"
+ ],
+ "metadata": {
+ "id": "XrJ2gQjfw1XP"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def gradient_descent_step(phi, data, model):\n",
+ " # Step 1: Compute the gradient\n",
+ " gradient = compute_gradient(data[0,:],data[1,:], phi)\n",
+ " # Step 2: Update the parameters -- note we want to search in the negative (downhill direction)\n",
+ " alpha = line_search(data, model, phi, gradient*-1, max_dist = 2.0)\n",
+ " phi = phi - alpha * gradient\n",
+ " return phi"
+ ],
+ "metadata": {
+ "id": "YVq6rmaWRD2M"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Initialize the parameters\n",
+ "n_steps = 21\n",
+ "phi_all = np.zeros((2,n_steps+1))\n",
+ "phi_all[0,0] = -1.5\n",
+ "phi_all[1,0] = 8.5\n",
+ "\n",
+ "# Measure loss and draw initial model\n",
+ "loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,0:1])\n",
+ "draw_model(data,model,phi_all[:,0:1], \"Initial parameters, Loss = %f\"%(loss))\n",
+ "\n",
+ "for c_step in range (n_steps):\n",
+ " # Do gradient descent step\n",
+ " phi_all[:,c_step+1:c_step+2] = gradient_descent_step(phi_all[:,c_step:c_step+1],data, model)\n",
+ " # Measure loss and draw model every 4th step\n",
+ " if c_step % 4 == 0:\n",
+ " loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,c_step+1:c_step+2])\n",
+ " draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
+ "\n",
+ "draw_loss_function(compute_loss, data, model,phi_all)\n"
+ ],
+ "metadata": {
+ "id": "tOLd0gtdRLLS"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO Experiment with starting the optimization in the previous cell in different places\n",
+ "# and show that it heads to a local minimum if we don't start it in the right valley"
+ ],
+ "metadata": {
+ "id": "Oi8ZlH0ptLqA"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def gradient_descent_step_fixed_learning_rate(phi, data, model, alpha):\n",
+ " # TODO -- fill in this routine so that we take a fixed size step of size alpha without using line search\n",
+ "\n",
+ " return phi"
+ ],
+ "metadata": {
+ "id": "4l-ueLk-oAxV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Initialize the parameters\n",
+ "n_steps = 21\n",
+ "phi_all = np.zeros((2,n_steps+1))\n",
+ "phi_all[0,0] = -1.5\n",
+ "phi_all[1,0] = 8.5\n",
+ "\n",
+ "# Measure loss and draw initial model\n",
+ "loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,0:1])\n",
+ "draw_model(data,model,phi_all[:,0:1], \"Initial parameters, Loss = %f\"%(loss))\n",
+ "\n",
+ "for c_step in range (n_steps):\n",
+ " # Do gradient descent step\n",
+ " phi_all[:,c_step+1:c_step+2] = gradient_descent_step_fixed_learning_rate(phi_all[:,c_step:c_step+1],data, model,alpha =0.2)\n",
+ " # Measure loss and draw model every 4th step\n",
+ " if c_step % 4 == 0:\n",
+ " loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,c_step+1:c_step+2])\n",
+ " draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
+ "\n",
+ "draw_loss_function(compute_loss, data, model,phi_all)\n"
+ ],
+ "metadata": {
+ "id": "oi9MX_GRpM41"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO Experiment with the learning rate, alpha. \n",
+ "# What happens if you set it too large?\n",
+ "# What happens if you set it too small?"
+ ],
+ "metadata": {
+ "id": "In6sQ5YCpMqn"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def stochastic_gradient_descent_step(phi, data, model, alpha, batch_size):\n",
+ " # TODO -- fill in this routine so that we take a fixed size step of size alpha but only using a subset (batch) of the data\n",
+ " # at each step\n",
+ " # You can use the function np.random.permutation to generate a random permutation of the n_data = data.shape[1] indices\n",
+ " # and then just choose the first n=batch_size of these indices. Then select compute the gradient update\n",
+ " # from just the data with these indices. Don't worry about sampling with replacement.\n",
+ "\n",
+ "\n",
+ " return phi"
+ ],
+ "metadata": {
+ "id": "VKTC9-1Gpm3N"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set the random number generator so you always get same numbers (disable if you don't want this)\n",
+ "np.random.seed(1)\n",
+ "# Initialize the parameters\n",
+ "n_steps = 41\n",
+ "phi_all = np.zeros((2,n_steps+1))\n",
+ "phi_all[0,0] = 3.5\n",
+ "phi_all[1,0] = 6.5\n",
+ "\n",
+ "# Measure loss and draw initial model\n",
+ "loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,0:1])\n",
+ "draw_model(data,model,phi_all[:,0:1], \"Initial parameters, Loss = %f\"%(loss))\n",
+ "\n",
+ "for c_step in range (n_steps):\n",
+ " # Do gradient descent step\n",
+ " phi_all[:,c_step+1:c_step+2] = stochastic_gradient_descent_step(phi_all[:,c_step:c_step+1],data, model,alpha =0.8, batch_size=5)\n",
+ " # Measure loss and draw model every 4th step\n",
+ " if c_step % 8 == 0:\n",
+ " loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,c_step+1:c_step+2])\n",
+ " draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
+ "\n",
+ "draw_loss_function(compute_loss, data, model,phi_all)"
+ ],
+ "metadata": {
+ "id": "469OP_UHskJ4"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO -- Experiment with different learning rates, starting points, batch sizes, number of steps. Get a feel for this."
+ ],
+ "metadata": {
+ "id": "LxE2kTa3s29p"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO -- How about adding a learning rate schedule? Reduce the learning rate by a factor of beta every M iterations"
+ ],
+ "metadata": {
+ "id": "lw4QPOaQTh5e"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
diff --git a/CM20315/CM20315_Transformers.ipynb b/CM20315/CM20315_Transformers.ipynb
new file mode 100644
index 0000000..7f1ecc2
--- /dev/null
+++ b/CM20315/CM20315_Transformers.ipynb
@@ -0,0 +1,634 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyMfWL40+ZshPZhweAtQ9Fn6",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Transformers\n",
+ "\n",
+ "This practical investigates neural decoding from transformer models. Run the next three cells as they might take a while to run (they have to download some stuff), and then read the next text box while you are waiting."
+ ],
+ "metadata": {
+ "id": "RnIUiieJWu6e"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "!pip install transformers"
+ ],
+ "metadata": {
+ "id": "7abjZ9pMVj3k"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "from transformers import GPT2LMHeadModel, GPT2Tokenizer, set_seed\n",
+ "import torch\n",
+ "import torch.nn.functional as F\n",
+ "import numpy as np"
+ ],
+ "metadata": {
+ "id": "sMOyD0zem2Ef"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Load model and tokenizer\n",
+ "model = GPT2LMHeadModel.from_pretrained('gpt2')\n",
+ "tokenizer = GPT2Tokenizer.from_pretrained('gpt2')"
+ ],
+ "metadata": {
+ "id": "pZgfxbzKWNSR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Decoding from GPT2\n",
+ "\n",
+ "This tutorial investigates how to use GPT2 (the forerunner of GPT3) to generate text. There are a number of ways to do this that trade-off the realism of the text against the amount of variation.\n",
+ "\n",
+ "At every stage, GPT2 takes an input string and returns a probability for each of the possible subsequent tokens. We can choose what to do with these probability. We could always *greedily choose* the most likely next token, or we could draw a *sample* randomly according to the probabilities. There are also intermediate strategies such as *top-k sampling* and *nucleus sampling*, that have some controlled randomness.\n",
+ "\n",
+ "We'll also investigate *beam search* -- the idea is that rather than greedily take the next best token at each stage, we maintain a set of hypotheses (beams)as we add each subsequent token and return the most likely overall hypothesis. This is not necessarily the same result we get from greedily choosing the next token. "
+ ],
+ "metadata": {
+ "id": "TfhAGy0TXEvV"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "First, let's investigate the token themselves. The code below prints out the vocabulary size and shows 20 random tokens. "
+ ],
+ "metadata": {
+ "id": "vsmO9ptzau3_"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "np.random.seed(1)\n",
+ "print(\"Number of tokens in dictionary = %d\"%(tokenizer.vocab_size))\n",
+ "for i in range(20):\n",
+ " index = np.random.randint(tokenizer.vocab_size)\n",
+ " print(\"Token: %d \"%(index)+tokenizer.decode(torch.tensor(index), skip_special_tokens=True))\n"
+ ],
+ "metadata": {
+ "id": "dmmBNS5GY_yk"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Sampling\n",
+ "\n",
+ "Each time we run GPT2 it will take in a set of tokens, and return a probability over each of the possible next tokens. The simplest thing we could do is to just draw a sample from this probability distribution each time."
+ ],
+ "metadata": {
+ "id": "MUM3kLEjbTso"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def sample_next_token(input_tokens, model, tokenizer):\n",
+ " # Run model to get prediction over next output\n",
+ " outputs = model(input_ids = input_tokens['input_ids'], attention_mask = input_tokens['attention_mask'])\n",
+ " # Find prediction\n",
+ " prob_over_tokens = F.softmax(outputs.logits, dim=-1).detach().numpy()[0,-1]\n",
+ " # TODO: Draw a random token according to the probabilities\n",
+ " # Use: https://numpy.org/doc/stable/reference/random/generated/numpy.random.choice.html\n",
+ " # Replace this line\n",
+ " next_token = [5000]\n",
+ "\n",
+ " # Append token to sentence\n",
+ " output_tokens = input_tokens\n",
+ " output_tokens[\"input_ids\"] = torch.cat((output_tokens['input_ids'],torch.tensor([next_token])),dim=1)\n",
+ " output_tokens['attention_mask'] = torch.cat((output_tokens['attention_mask'],torch.tensor([[1]])),dim=1)\n",
+ " output_tokens['last_token_prob'] = prob_over_tokens[next_token]\n",
+ "\n",
+ " return output_tokens"
+ ],
+ "metadata": {
+ "id": "TIyNgg0FkJKO"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Expected output:\n",
+ "# \"The best thing about Bath is that they don't even change or shrink anymore.\"\n",
+ "\n",
+ "set_seed(0)\n",
+ "input_txt = \"The best thing about Bath is\"\n",
+ "input_tokens = tokenizer(input_txt, return_tensors='pt')\n",
+ "for i in range(10):\n",
+ " input_tokens = sample_next_token(input_tokens, model, tokenizer)\n",
+ " print(tokenizer.decode(input_tokens[\"input_ids\"][0], skip_special_tokens=True))\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "BHs-IWaz9MNY"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO Modify the code below by changing the number of tokens generated and the initial sentence\n",
+ "# to get a feel for how well this works. Since I didn't reset the seed, it will give a different\n",
+ "# answer every time that you run it.\n",
+ "\n",
+ "# TODO Experiment with changing this line:\n",
+ "input_txt = \"The best thing about Bath is\"\n",
+ "input_tokens = tokenizer(input_txt, return_tensors='pt')\n",
+ "# TODO Experiment with changing this line:\n",
+ "for i in range(10):\n",
+ " input_tokens = sample_next_token(input_tokens, model, tokenizer)\n",
+ " print(tokenizer.decode(input_tokens[\"input_ids\"][0], skip_special_tokens=True))"
+ ],
+ "metadata": {
+ "id": "yN98_7WqbvIe"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Greedy token selection\n",
+ "\n",
+ "You probably (correctly) got the impression that the text from pure sampling of the probability model can be kind of random. How about if we choose most likely token at each step?\n"
+ ],
+ "metadata": {
+ "id": "7eHFLCeZcmmg"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def get_best_next_token(input_tokens, model, tokenizer):\n",
+ " # Run model to get prediction over next output\n",
+ " outputs = model(input_ids = input_tokens['input_ids'], attention_mask = input_tokens['attention_mask'])\n",
+ " # Find prediction\n",
+ " prob_over_tokens = F.softmax(outputs.logits, dim=-1).detach().numpy()[0,-1]\n",
+ "\n",
+ " # TODO -- find the token index with the maximum probability\n",
+ " # It should be returns as a list (i.e., put squared brackets around it)\n",
+ " # Use https://numpy.org/doc/stable/reference/generated/numpy.argmax.html\n",
+ " # Replace this line\n",
+ " next_token = [5000]\n",
+ "\n",
+ " # Append token to sentence\n",
+ " output_tokens = input_tokens\n",
+ " output_tokens[\"input_ids\"] = torch.cat((output_tokens['input_ids'],torch.tensor([next_token])),dim=1)\n",
+ " output_tokens['attention_mask'] = torch.cat((output_tokens['attention_mask'],torch.tensor([[1]])),dim=1)\n",
+ " output_tokens['last_token_prob'] = prob_over_tokens[next_token]\n",
+ " return output_tokens"
+ ],
+ "metadata": {
+ "id": "OhRzynEjxpZF"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Expected output:\n",
+ "# The best thing about Bath is that it's a place where you can go to\n",
+ "set_seed(0)\n",
+ "input_txt = \"The best thing about Bath is\"\n",
+ "input_tokens = tokenizer(input_txt, return_tensors='pt')\n",
+ "for i in range(10):\n",
+ " input_tokens = get_best_next_token(input_tokens, model, tokenizer)\n",
+ " print(tokenizer.decode(input_tokens[\"input_ids\"][0], skip_special_tokens=True))"
+ ],
+ "metadata": {
+ "id": "gKB1Mgndj-Hm"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO Modify the code below by changing the number of tokens generated and the initial sentence\n",
+ "# to get a feel for how well this works. \n",
+ "\n",
+ "# TODO Experiment with changing this line:\n",
+ "input_txt = \"The best thing about Bath is\"\n",
+ "input_tokens = tokenizer(input_txt, return_tensors='pt')\n",
+ "# TODO Experiment with changing this line:\n",
+ "for i in range(10):\n",
+ " input_tokens = get_best_next_token(input_tokens, model, tokenizer)\n",
+ " print(tokenizer.decode(input_tokens[\"input_ids\"][0], skip_special_tokens=True))"
+ ],
+ "metadata": {
+ "id": "L1YHKaYFfC0M"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Top-K sampling\n",
+ "\n",
+ "You probably noticed that the greedy strategy produces quite realistic text, but it's kind of boring. It produces generic answers. Also, if this was a chatbot, then we wouldn't necessarily want it to produce the same answer to a question each time. \n",
+ "\n",
+ "Top-K sampling is a compromise strategy that samples randomly from the top K most probable tokens. We could just choose them with a uniform distribution, or (as here) we could sample them according to their original probabilities."
+ ],
+ "metadata": {
+ "id": "1ORFXYX_gBDT"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def get_top_k_token(input_tokens, model, tokenizer, k=20):\n",
+ " # Run model to get prediction over next output\n",
+ " outputs = model(input_ids = input_tokens['input_ids'], attention_mask = input_tokens['attention_mask'])\n",
+ " # Find prediction\n",
+ " prob_over_tokens = F.softmax(outputs.logits, dim=-1).detach().numpy()[0,-1]\n",
+ "\n",
+ " # Draw a sample from the top K most likely tokens.\n",
+ " # Take copy of the probabilities and sort from largest to smallest (use np.sort)\n",
+ " # TODO -- replace this line\n",
+ " sorted_prob_over_tokens = prob_over_tokens\n",
+ "\n",
+ " # Find the probability at the k'th position\n",
+ " # TODO -- replace this line\n",
+ " kth_prob_value = 0.0\n",
+ "\n",
+ " # Set all probabilities below this value to zero \n",
+ " prob_over_tokens[prob_over_tokens"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Coursework I -- Model hyperparameters\n",
+ "\n",
+ "The goal of the coursework is to modify a simple bit of numpy code that trains a network and measures the performance on a validation set for the MNIST 1D dataset.\n",
+ "\n",
+ "In this coursework, you need to modify the **model hyperparameters** (only) to improve the performance over the current attempt. This could mean the number of layers, the number of hidden units per layer, or the type of activation function, or any combination of the three.\n",
+ "\n",
+ "The only constraint is that you MUST use a fully connected network (no convolutional networks for now if you have read ahead in the book).\n",
+ "\n",
+ "You must improve the performance by at least 2% to get full marks.\n",
+ "\n",
+ "You will need to upload three things to Moodle:\n",
+ "1. The image that this notebook saves (click the folder icon on the left on colab to download it)\n",
+ "2. The lines of code you changed\n",
+ "3. The whole notebook as a .ipynb file. You can do this on the File menu\n",
+ "\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import os\n",
+ "import torch, torch.nn as nn\n",
+ "from torch.utils.data import TensorDataset, DataLoader\n",
+ "from torch.optim.lr_scheduler import StepLR\n",
+ "import matplotlib.pyplot as plt\n",
+ "import random\n",
+ "import gdown"
+ ],
+ "metadata": {
+ "id": "YrXWAH7sUWvU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "\n",
+ "# Run this once to copy the train and validation data to your CoLab environment\n",
+ "# or download from my github to your local machine if you are doing this locally\n",
+ "if not os.path.exists('./Data.zip'):\n",
+ " !gdown 1HtnCrncY6dFCYqzgPf1HtPVAerTpwFRm\n",
+ " !unzip Data.zip"
+ ],
+ "metadata": {
+ "id": "wScBGXXFVadm"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Load in the data\n",
+ "train_data_x = np.load('train_data_x.npy')\n",
+ "val_data_y = np.load('val_data_y.npy')\n",
+ "train_data_y = np.load('train_data_y.npy')\n",
+ "val_data_x = np.load('val_data_x.npy')\n",
+ "# Print out sizes\n",
+ "print(\"Train data: %d examples (columns), each of which has %d dimensions (rows)\"%((train_data_x.shape[1],train_data_x.shape[0])))\n",
+ "print(\"Validation data: %d examples (columns), each of which has %d dimensions (rows)\"%((val_data_x.shape[1],val_data_x.shape[0])))"
+ ],
+ "metadata": {
+ "id": "8bKADvLHbiV5"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define the network"
+ ],
+ "metadata": {
+ "id": "_sFvRDGrl4qe"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# YOU SHOULD ONLY CHANGE THIS CELL!\n",
+ "\n",
+ "# There are 40 input dimensions and 10 output dimensions for this data\n",
+ "# The inputs correspond to the 40 offsets in the MNIST1D template.\n",
+ "D_i = 40\n",
+ "# The outputs correspond to the 10 digits\n",
+ "D_o = 10\n",
+ "\n",
+ "# Number of hidden units in layers 1 and 2\n",
+ "D_1 = 100\n",
+ "D_2 = 100\n",
+ "\n",
+ "# create model with two hidden layers\n",
+ "model = nn.Sequential(\n",
+ "nn.Linear(D_i, D_1),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(D_1, D_2),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(D_2, D_o))"
+ ],
+ "metadata": {
+ "id": "FslroPJJffrh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# He initialization of weights\n",
+ "def weights_init(layer_in):\n",
+ " if isinstance(layer_in, nn.Linear):\n",
+ " nn.init.kaiming_uniform_(layer_in.weight)\n",
+ " layer_in.bias.data.fill_(0.0)"
+ ],
+ "metadata": {
+ "id": "YgLaex1pfhqz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# You need all this stuff to ensure that PyTorch is deterministic\n",
+ "def set_seed(seed):\n",
+ " torch.manual_seed(seed)\n",
+ " torch.cuda.manual_seed_all(seed)\n",
+ " torch.backends.cudnn.deterministic = True\n",
+ " torch.backends.cudnn.benchmark = False\n",
+ " np.random.seed(seed)\n",
+ " random.seed(seed)\n",
+ " os.environ['PYTHONHASHSEED'] = str(seed)"
+ ],
+ "metadata": {
+ "id": "zXRmxCQNnL_M"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set seed so always get same result (do not change)\n",
+ "set_seed(1)\n",
+ "\n",
+ "# choose cross entropy loss function (equation 5.24 in the loss notes)\n",
+ "loss_function = nn.CrossEntropyLoss()\n",
+ "# construct SGD optimizer and initialize learning rate and momentum\n",
+ "optimizer = torch.optim.SGD(model.parameters(), lr = 0.05, momentum=0.9)\n",
+ "# object that decreases learning rate by half every 10 epochs\n",
+ "scheduler = StepLR(optimizer, step_size=10, gamma=0.5)\n",
+ "# create 100 dummy data points and store in data loader class\n",
+ "x_train = torch.tensor(train_data_x.transpose().astype('float32'))\n",
+ "y_train = torch.tensor(train_data_y.astype('long'))\n",
+ "x_val= torch.tensor(val_data_x.transpose().astype('float32'))\n",
+ "y_val = torch.tensor(val_data_y.astype('long'))\n",
+ "\n",
+ "# load the data into a class that creates the batches\n",
+ "data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
+ "\n",
+ "# Initialize model weights\n",
+ "model.apply(weights_init)\n",
+ "\n",
+ "# loop over the dataset n_epoch times\n",
+ "n_epoch = 50\n",
+ "# store the loss and the % correct at each epoch\n",
+ "losses_train = np.zeros((n_epoch))\n",
+ "errors_train = np.zeros((n_epoch))\n",
+ "losses_val = np.zeros((n_epoch))\n",
+ "errors_val = np.zeros((n_epoch))\n",
+ "\n",
+ "for epoch in range(n_epoch):\n",
+ " # loop over batches\n",
+ " for i, data in enumerate(data_loader):\n",
+ " # retrieve inputs and labels for this batch\n",
+ " x_batch, y_batch = data\n",
+ " # zero the parameter gradients\n",
+ " optimizer.zero_grad()\n",
+ " # forward pass -- calculate model output\n",
+ " pred = model(x_batch)\n",
+ " # compute the lss\n",
+ " loss = loss_function(pred, y_batch)\n",
+ " # backward pass\n",
+ " loss.backward()\n",
+ " # SGD update\n",
+ " optimizer.step()\n",
+ "\n",
+ " # Run whole dataset to get statistics -- normally wouldn't do this\n",
+ " pred_train = model(x_train)\n",
+ " pred_val = model(x_val)\n",
+ " _, predicted_train_class = torch.max(pred_train.data, 1)\n",
+ " _, predicted_val_class = torch.max(pred_val.data, 1)\n",
+ " errors_train[epoch] = 100 - 100 * (predicted_train_class == y_train).float().sum() / len(y_train)\n",
+ " errors_val[epoch]= 100 - 100 * (predicted_val_class == y_val).float().sum() / len(y_val)\n",
+ " losses_train[epoch] = loss_function(pred_train, y_train).item()\n",
+ " losses_val[epoch]= loss_function(pred_val, y_val).item()\n",
+ " print(f'Epoch {epoch:5d}, train loss {losses_train[epoch]:.6f}, train error {errors_train[epoch]:3.2f}, val loss {losses_val[epoch]:.6f}, percent error {errors_val[epoch]:3.2f}')\n",
+ "\n",
+ " # tell scheduler to consider updating learning rate\n",
+ " scheduler.step()\n",
+ "\n",
+ "# Plot the results\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(errors_train,'r-',label='train')\n",
+ "ax.plot(errors_val,'b-',label='validation')\n",
+ "ax.set_ylim(0,100); ax.set_xlim(0,n_epoch)\n",
+ "ax.set_xlabel('Epoch'); ax.set_ylabel('Error')\n",
+ "ax.set_title('Part I: Validation Result %3.2f'%(errors_val[-1]))\n",
+ "ax.legend()\n",
+ "ax.plot([0,n_epoch],[37.45, 37.45],'k:') # Original results. You should be better than this!\n",
+ "plt.savefig('Coursework_I_Results.png',format='png')\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "NYw8I_3mmX5c"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Leave this all commented for now\n",
+ "# We'll see how well you did on the test data after the coursework is submitted\n",
+ "\n",
+ "# # I haven't given you this yet, leave commented\n",
+ "# test_data_x = np.load('test_data_x.npy')\n",
+ "# test_data_y = np.load('test_data_y.npy')\n",
+ "# x_test = torch.tensor(test_data_x.transpose().astype('float32'))\n",
+ "# y_test = torch.tensor(test_data_y.astype('long'))\n",
+ "# pred_test = model(x_test)\n",
+ "# _, predicted_test_class = torch.max(pred_test.data, 1)\n",
+ "# errors_test = 100 - 100 * (predicted_test_class == y_test).float().sum() / len(y_test)\n",
+ "# print(\"Test error = %3.3f\"%(errors_test))"
+ ],
+ "metadata": {
+ "id": "O7nBz-R84QdJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/CM20315_2023/CM20315_Coursework_II.ipynb b/CM20315_2023/CM20315_Coursework_II.ipynb
new file mode 100644
index 0000000..0ce1584
--- /dev/null
+++ b/CM20315_2023/CM20315_Coursework_II.ipynb
@@ -0,0 +1,276 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyM+iKos5DEoHUxL8+9oxA2A",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Coursework II -- Training hyperparameters\n",
+ "\n",
+ "The goal of the coursework is to modify a simple bit of numpy code that trains a network and measures the performance on a validation set for the MNist 1D dataset.\n",
+ "\n",
+ "In this coursework, you need to modify the **training hyperparameters** (only) to improve the performance over the current attempt. This could mean the training algorithm, learning rate, learning rate schedule, momentum term, initialization etc. \n",
+ "\n",
+ "You must improve the performance by at least 2% to get full marks.\n",
+ "\n",
+ "You will need to upload three things to Moodle:\n",
+ "1. The image that this notebook saves (click the folder icon on the left on colab to download it)\n",
+ "2. The lines of code you changed\n",
+ "3. The whole notebook as a .ipynb file. You can do this on the File menu"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import os\n",
+ "import torch, torch.nn as nn\n",
+ "from torch.utils.data import TensorDataset, DataLoader\n",
+ "from torch.optim.lr_scheduler import StepLR\n",
+ "import matplotlib.pyplot as plt\n",
+ "import random\n",
+ "import gdown"
+ ],
+ "metadata": {
+ "id": "YrXWAH7sUWvU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run this once to copy the train and validation data to your CoLab environment\n",
+ "if not os.path.exists('./Data.zip'):\n",
+ " !gdown 1HtnCrncY6dFCYqzgPf1HtPVAerTpwFRm\n",
+ " !unzip Data.zip"
+ ],
+ "metadata": {
+ "id": "wScBGXXFVadm"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Load in the data\n",
+ "train_data_x = np.load('train_data_x.npy',allow_pickle=True)\n",
+ "train_data_y = np.load('train_data_y.npy',allow_pickle=True)\n",
+ "val_data_x = np.load('val_data_x.npy',allow_pickle=True)\n",
+ "val_data_y = np.load('val_data_y.npy',allow_pickle=True)\n",
+ "# Print out sizes\n",
+ "print(\"Train data: %d examples (columns), each of which has %d dimensions (rows)\"%((train_data_x.shape[1],train_data_x.shape[0])))\n",
+ "print(\"Validation data: %d examples (columns), each of which has %d dimensions (rows)\"%((val_data_x.shape[1],val_data_x.shape[0])))"
+ ],
+ "metadata": {
+ "id": "8bKADvLHbiV5"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define the network"
+ ],
+ "metadata": {
+ "id": "_sFvRDGrl4qe"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# YOU SHOULD NOT CHANGE THIS CELL!\n",
+ "\n",
+ "# There are 40 input dimensions and 10 output dimensions for this data\n",
+ "# The inputs correspond to the 40 offsets in the MNIST1D template.\n",
+ "D_i = 40\n",
+ "# The outputs correspond to the 10 digits\n",
+ "D_o = 10\n",
+ "\n",
+ "# Number of hidden units in layers 1 and 2\n",
+ "D_1 = 100\n",
+ "D_2 = 100\n",
+ "\n",
+ "# create model with two hidden layers\n",
+ "model = nn.Sequential(\n",
+ "nn.Linear(D_i, D_1),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(D_1, D_2),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(D_2, D_o))"
+ ],
+ "metadata": {
+ "id": "FslroPJJffrh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# He initialization of weights\n",
+ "def weights_init(layer_in):\n",
+ " if isinstance(layer_in, nn.Linear):\n",
+ " nn.init.kaiming_uniform_(layer_in.weight)\n",
+ " layer_in.bias.data.fill_(0.0)"
+ ],
+ "metadata": {
+ "id": "YgLaex1pfhqz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# You need all this stuff to ensure that PyTorch is deterministic\n",
+ "def set_seed(seed):\n",
+ " torch.manual_seed(seed)\n",
+ " torch.cuda.manual_seed_all(seed)\n",
+ " torch.backends.cudnn.deterministic = True\n",
+ " torch.backends.cudnn.benchmark = False\n",
+ " np.random.seed(seed)\n",
+ " random.seed(seed)\n",
+ " os.environ['PYTHONHASHSEED'] = str(seed)"
+ ],
+ "metadata": {
+ "id": "zXRmxCQNnL_M"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set seed so always get same result (do not change)\n",
+ "set_seed(1)\n",
+ "\n",
+ "# choose cross entropy loss function (equation 5.24 in the loss notes)\n",
+ "loss_function = nn.CrossEntropyLoss()\n",
+ "# construct SGD optimizer and initialize learning rate and momentum\n",
+ "optimizer = torch.optim.SGD(model.parameters(), lr = 0.05, momentum=0.9)\n",
+ "# object that decreases learning rate by half every 10 epochs\n",
+ "scheduler = StepLR(optimizer, step_size=10, gamma=0.5)\n",
+ "# create 100 dummy data points and store in data loader class\n",
+ "x_train = torch.tensor(train_data_x.transpose().astype('float32'))\n",
+ "print(x_train.shape)\n",
+ "y_train = torch.tensor(train_data_y.astype('long'))\n",
+ "print(y_train.shape)\n",
+ "x_val= torch.tensor(val_data_x.transpose().astype('float32'))\n",
+ "y_val = torch.tensor(val_data_y.astype('long'))\n",
+ "\n",
+ "# load the data into a class that creates the batches\n",
+ "data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
+ "\n",
+ "# Initialize model weights\n",
+ "model.apply(weights_init)\n",
+ "\n",
+ "# loop over the dataset n_epoch times\n",
+ "n_epoch = 50\n",
+ "# store the loss and the % correct at each epoch\n",
+ "losses_train = np.zeros((n_epoch))\n",
+ "errors_train = np.zeros((n_epoch))\n",
+ "losses_val = np.zeros((n_epoch))\n",
+ "errors_val = np.zeros((n_epoch))\n",
+ "\n",
+ "for epoch in range(n_epoch):\n",
+ " # loop over batches\n",
+ " for i, data in enumerate(data_loader):\n",
+ " # retrieve inputs and labels for this batch\n",
+ " x_batch, y_batch = data\n",
+ " # zero the parameter gradients\n",
+ " optimizer.zero_grad()\n",
+ " # forward pass -- calculate model output\n",
+ " pred = model(x_batch)\n",
+ " # compute the lss\n",
+ " loss = loss_function(pred, y_batch)\n",
+ " # backward pass\n",
+ " loss.backward()\n",
+ " # SGD update\n",
+ " optimizer.step()\n",
+ "\n",
+ " # Run whole dataset to get statistics -- normally wouldn't do this\n",
+ " pred_train = model(x_train)\n",
+ " pred_val = model(x_val)\n",
+ " _, predicted_train_class = torch.max(pred_train.data, 1)\n",
+ " _, predicted_val_class = torch.max(pred_val.data, 1)\n",
+ " errors_train[epoch] = 100 - 100 * (predicted_train_class == y_train).float().sum() / len(y_train)\n",
+ " errors_val[epoch]= 100 - 100 * (predicted_val_class == y_val).float().sum() / len(y_val)\n",
+ " losses_train[epoch] = loss_function(pred_train, y_train).item()\n",
+ " losses_val[epoch]= loss_function(pred_val, y_val).item()\n",
+ " print(f'Epoch {epoch:5d}, train loss {losses_train[epoch]:.6f}, train error {errors_train[epoch]:3.2f}, val loss {losses_val[epoch]:.6f}, percent error {errors_val[epoch]:3.2f}')\n",
+ "\n",
+ " # tell scheduler to consider updating learning rate\n",
+ " scheduler.step()\n",
+ "\n",
+ "# Plot the results\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(errors_train,'r-',label='train')\n",
+ "ax.plot(errors_val,'b-',label='validation')\n",
+ "ax.set_ylim(0,100); ax.set_xlim(0,n_epoch)\n",
+ "ax.set_xlabel('Epoch'); ax.set_ylabel('Error')\n",
+ "ax.set_title('Part II: Validation Result %3.2f'%(errors_val[-1]))\n",
+ "ax.legend()\n",
+ "ax.plot([0,n_epoch],[37.45, 37.45],'k:') # Original results. You should be better than this!\n",
+ "plt.savefig('Coursework_II_Results.png',format='png')\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "NYw8I_3mmX5c"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Leave this all commented for now\n",
+ "# We'll see how well you did on the test data after the coursework is submitted\n",
+ "\n",
+ "# # I haven't given you this yet, leave commented\n",
+ "# test_data_x = np.load('test_data_x.npy')\n",
+ "# test_data_y = np.load('test_data_y.npy')\n",
+ "# x_test = torch.tensor(test_data_x.transpose().astype('float32'))\n",
+ "# y_test = torch.tensor(test_data_y.astype('long'))\n",
+ "# pred_test = model(x_test)\n",
+ "# _, predicted_test_class = torch.max(pred_test.data, 1)\n",
+ "# errors_test = 100 - 100 * (predicted_test_class == y_test).float().sum() / len(y_test)\n",
+ "# print(\"Test error = %3.3f\"%(errors_test))"
+ ],
+ "metadata": {
+ "id": "O7nBz-R84QdJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/CM20315_2023/CM20315_Coursework_III.ipynb b/CM20315_2023/CM20315_Coursework_III.ipynb
new file mode 100644
index 0000000..ebc7cdf
--- /dev/null
+++ b/CM20315_2023/CM20315_Coursework_III.ipynb
@@ -0,0 +1,275 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyNDH1z3I76jjglu3o0LSlZc",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Coursework III -- Regularization\n",
+ "\n",
+ "The goal of the coursework is to modify a simple bit of numpy code that trains a network and measures the performance on a validation set for the MNist 1D dataset.\n",
+ "\n",
+ "In this coursework, you need add **regularization** of some kind to improve the performance. Anything from chapter 9 of the book or anything else you can find is fine *except* early stopping. You must not change the model hyperparameters or the training algorithm.\n",
+ "\n",
+ "You must improve the performance by at least 2% to get full marks.\n",
+ "\n",
+ "You will need to upload three things to Moodle:\n",
+ "1. The image that this notebook saves (click the folder icon on the left on colab to download it)\n",
+ "2. The lines of code you changed\n",
+ "3. The whole notebook as a .ipynb file. You can do this on the File menu\n",
+ "\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import os\n",
+ "import torch, torch.nn as nn\n",
+ "from torch.utils.data import TensorDataset, DataLoader\n",
+ "from torch.optim.lr_scheduler import StepLR\n",
+ "import matplotlib.pyplot as plt\n",
+ "import random\n",
+ "import gdown"
+ ],
+ "metadata": {
+ "id": "YrXWAH7sUWvU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run this once to copy the train and validation data to your CoLab environment\n",
+ "if not os.path.exists('./Data.zip'):\n",
+ " !gdown 1HtnCrncY6dFCYqzgPf1HtPVAerTpwFRm\n",
+ " !unzip Data.zip"
+ ],
+ "metadata": {
+ "id": "wScBGXXFVadm"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Load in the data\n",
+ "train_data_x = np.load('train_data_x.npy')\n",
+ "train_data_y = np.load('train_data_y.npy')\n",
+ "val_data_x = np.load('val_data_x.npy')\n",
+ "val_data_y = np.load('val_data_y.npy')\n",
+ "# Print out sizes\n",
+ "print(\"Train data: %d examples (columns), each of which has %d dimensions (rows)\"%((train_data_x.shape[1],train_data_x.shape[0])))\n",
+ "print(\"Validation data: %d examples (columns), each of which has %d dimensions (rows)\"%((val_data_x.shape[1],val_data_x.shape[0])))"
+ ],
+ "metadata": {
+ "id": "8bKADvLHbiV5"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define the network"
+ ],
+ "metadata": {
+ "id": "_sFvRDGrl4qe"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# There are 40 input dimensions and 10 output dimensions for this data\n",
+ "# The inputs correspond to the 40 offsets in the MNIST1D template.\n",
+ "D_i = 40\n",
+ "# The outputs correspond to the 10 digits\n",
+ "D_o = 10\n",
+ "\n",
+ "# Number of hidden units in layers 1 and 2\n",
+ "D_1 = 100\n",
+ "D_2 = 100\n",
+ "\n",
+ "# create model with two hidden layers\n",
+ "model = nn.Sequential(\n",
+ "nn.Linear(D_i, D_1),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(D_1, D_2),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(D_2, D_o))"
+ ],
+ "metadata": {
+ "id": "FslroPJJffrh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# He initialization of weights\n",
+ "def weights_init(layer_in):\n",
+ " if isinstance(layer_in, nn.Linear):\n",
+ " nn.init.kaiming_uniform_(layer_in.weight)\n",
+ " layer_in.bias.data.fill_(0.0)"
+ ],
+ "metadata": {
+ "id": "YgLaex1pfhqz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# You need all this stuff to ensure that PyTorch is deterministic\n",
+ "def set_seed(seed):\n",
+ " torch.manual_seed(seed)\n",
+ " torch.cuda.manual_seed_all(seed)\n",
+ " torch.backends.cudnn.deterministic = True\n",
+ " torch.backends.cudnn.benchmark = False\n",
+ " np.random.seed(seed)\n",
+ " random.seed(seed)\n",
+ " os.environ['PYTHONHASHSEED'] = str(seed)"
+ ],
+ "metadata": {
+ "id": "zXRmxCQNnL_M"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set seed so always get same result (do not change)\n",
+ "set_seed(1)\n",
+ "\n",
+ "# choose cross entropy loss function (equation 5.24 in the loss notes)\n",
+ "loss_function = nn.CrossEntropyLoss()\n",
+ "# construct SGD optimizer and initialize learning rate and momentum\n",
+ "optimizer = torch.optim.SGD(model.parameters(), lr = 0.05, momentum=0.9)\n",
+ "# object that decreases learning rate by half every 10 epochs\n",
+ "scheduler = StepLR(optimizer, step_size=10, gamma=0.5)\n",
+ "# create 100 dummy data points and store in data loader class\n",
+ "x_train = torch.tensor(train_data_x.transpose().astype('float32'))\n",
+ "y_train = torch.tensor(train_data_y.astype('long'))\n",
+ "x_val= torch.tensor(val_data_x.transpose().astype('float32'))\n",
+ "y_val = torch.tensor(val_data_y.astype('long'))\n",
+ "\n",
+ "# load the data into a class that creates the batches\n",
+ "data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
+ "\n",
+ "# Initialize model weights\n",
+ "model.apply(weights_init)\n",
+ "\n",
+ "# loop over the dataset n_epoch times\n",
+ "n_epoch = 50\n",
+ "# store the loss and the % correct at each epoch\n",
+ "losses_train = np.zeros((n_epoch))\n",
+ "errors_train = np.zeros((n_epoch))\n",
+ "losses_val = np.zeros((n_epoch))\n",
+ "errors_val = np.zeros((n_epoch))\n",
+ "\n",
+ "for epoch in range(n_epoch):\n",
+ " # loop over batches\n",
+ " for i, data in enumerate(data_loader):\n",
+ " # retrieve inputs and labels for this batch\n",
+ " x_batch, y_batch = data\n",
+ " # zero the parameter gradients\n",
+ " optimizer.zero_grad()\n",
+ " # forward pass -- calculate model output\n",
+ " pred = model(x_batch)\n",
+ " # compute the lss\n",
+ " loss = loss_function(pred, y_batch)\n",
+ " # backward pass\n",
+ " loss.backward()\n",
+ " # SGD update\n",
+ " optimizer.step()\n",
+ "\n",
+ " # Run whole dataset to get statistics -- normally wouldn't do this\n",
+ " pred_train = model(x_train)\n",
+ " pred_val = model(x_val)\n",
+ " _, predicted_train_class = torch.max(pred_train.data, 1)\n",
+ " _, predicted_val_class = torch.max(pred_val.data, 1)\n",
+ " errors_train[epoch] = 100 - 100 * (predicted_train_class == y_train).float().sum() / len(y_train)\n",
+ " errors_val[epoch]= 100 - 100 * (predicted_val_class == y_val).float().sum() / len(y_val)\n",
+ " losses_train[epoch] = loss_function(pred_train, y_train).item()\n",
+ " losses_val[epoch]= loss_function(pred_val, y_val).item()\n",
+ " print(f'Epoch {epoch:5d}, train loss {losses_train[epoch]:.6f}, train error {errors_train[epoch]:3.2f}, val loss {losses_val[epoch]:.6f}, percent error {errors_val[epoch]:3.2f}')\n",
+ "\n",
+ " # tell scheduler to consider updating learning rate\n",
+ " scheduler.step()\n",
+ "\n",
+ "# Plot the results\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(errors_train,'r-',label='train')\n",
+ "ax.plot(errors_val,'b-',label='validation')\n",
+ "ax.set_ylim(0,100); ax.set_xlim(0,n_epoch)\n",
+ "ax.set_xlabel('Epoch'); ax.set_ylabel('Error')\n",
+ "ax.set_title('Part III: Validation Result %3.2f'%(errors_val[-1]))\n",
+ "ax.legend()\n",
+ "ax.plot([0,n_epoch],[37.45, 37.45],'k:') # Original results. You should be better than this!\n",
+ "plt.savefig('Coursework_III_Results.png',format='png')\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "NYw8I_3mmX5c"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Leave this all commented for now\n",
+ "# We'll see how well you did on the test data after the coursework is submitted\n",
+ "\n",
+ "\n",
+ "# # I haven't given you this yet, leave commented\n",
+ "# test_data_x = np.load('test_data_x.npy')\n",
+ "# test_data_y = np.load('test_data_y.npy')\n",
+ "# x_test = torch.tensor(test_data_x.transpose().astype('float32'))\n",
+ "# y_test = torch.tensor(test_data_y.astype('long'))\n",
+ "# pred_test = model(x_test)\n",
+ "# _, predicted_test_class = torch.max(pred_test.data, 1)\n",
+ "# errors_test = 100 - 100 * (predicted_test_class == y_test).float().sum() / len(y_test)\n",
+ "# print(\"Test error = %3.3f\"%(errors_test))"
+ ],
+ "metadata": {
+ "id": "O7nBz-R84QdJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/CM20315_2023/CM20315_Coursework_IV.ipynb b/CM20315_2023/CM20315_Coursework_IV.ipynb
new file mode 100644
index 0000000..8f98ad5
--- /dev/null
+++ b/CM20315_2023/CM20315_Coursework_IV.ipynb
@@ -0,0 +1,212 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyMrWYwQrwgJvDza1vhYK9WQ",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Coursework IV\n",
+ "\n",
+ "This coursework explores the geometry of high dimensional spaces. It doesn't behave how you would expect and all your intuitions are wrong! You will write code and it will give you three numerical answers that you need to type into Moodle."
+ ],
+ "metadata": {
+ "id": "EjLK-kA1KnYX"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4ESMmnkYEVAb"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "import scipy.special as sci"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Part (a)\n",
+ "\n",
+ "In part (a) of the practical, we investigate how close random points are in 2D, 100D, and 1000D. In each case, we generate 1000 points and calculate the Euclidean distance between each pair. You should find that in 1000D, the furthest two points are only slightly further apart than the nearest points. Weird!"
+ ],
+ "metadata": {
+ "id": "MonbPEitLNgN"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Fix the random seed so we all have the same random numbers\n",
+ "np.random.seed(0)\n",
+ "n_data = 1000\n",
+ "# Create 1000 data examples (columns) each with 2 dimensions (rows)\n",
+ "n_dim = 2\n",
+ "x_2D = np.random.normal(size=(n_dim,n_data))\n",
+ "# Create 1000 data examples (columns) each with 100 dimensions (rows)\n",
+ "n_dim = 100\n",
+ "x_100D = np.random.normal(size=(n_dim,n_data))\n",
+ "# Create 1000 data examples (columns) each with 1000 dimensions (rows)\n",
+ "n_dim = 1000\n",
+ "x_1000D = np.random.normal(size=(n_dim,n_data))\n",
+ "\n",
+ "# These values should be the same, otherwise your answer will be wrong\n",
+ "# Get in touch if they are not!\n",
+ "print('Sum of your data is %3.3f, Should be %3.3f'%(np.sum(x_1000D),1036.321))"
+ ],
+ "metadata": {
+ "id": "vZSHVmcWEk14"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def distance_ratio(x):\n",
+ " # TODO -- replace the two lines below to calculate the largest and smallest Euclidean distance between\n",
+ " # the data points in the columns of x. DO NOT include the distance between the data point\n",
+ " # and itself (which is obviously zero)\n",
+ " smallest_dist = 1.0\n",
+ " largest_dist = 1.0\n",
+ "\n",
+ " # Calculate the ratio and return\n",
+ " dist_ratio = largest_dist / smallest_dist\n",
+ " return dist_ratio"
+ ],
+ "metadata": {
+ "id": "PhVmnUs8ErD9"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "print('Ratio of largest to smallest distance 2D: %3.3f'%(distance_ratio(x_2D)))\n",
+ "print('Ratio of largest to smallest distance 100D: %3.3f'%(distance_ratio(x_100D)))\n",
+ "print('Ratio of largest to smallest distance 1000D: %3.3f'%(distance_ratio(x_1000D)))\n",
+ "print('**Note down the last of these three numbers, you will need to submit it for your coursework**')"
+ ],
+ "metadata": {
+ "id": "0NdPxfn5GQuJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Part (b)\n",
+ "\n",
+ "In part (b) of the practical we calculate the volume of a hypersphere of radius 0.5 (i.e., of diameter 1) as a function of the radius. You will find that the volume decreases to almost nothing in high dimensions. All of the volume is in the corners of the unit hypercube (which always has volume 1). Double weird.\n",
+ "\n",
+ "Note that you you can check your answer by doing the calculation for 2D using the standard formula for the area of a circle and making sure it matches."
+ ],
+ "metadata": {
+ "id": "b2FYKV1SL4Z7"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def volume_of_hypersphere(diameter, dimensions):\n",
+ " # Formula given in Problem 8.7 of the notes\n",
+ " # You will need sci.gamma()\n",
+ " # Check out: https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.gamma.html\n",
+ " # Also use this value for pi\n",
+ " pi = np.pi\n",
+ " # TODO replace this code with formula for the volume of a hypersphere\n",
+ " volume = 1.0\n",
+ "\n",
+ " return volume\n"
+ ],
+ "metadata": {
+ "id": "CZoNhD8XJaHR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "diameter = 1.0\n",
+ "for c_dim in range(1,11):\n",
+ " print(\"Volume of unit diameter hypersphere in %d dimensions is %3.3f\"%(c_dim, volume_of_hypersphere(diameter, c_dim)))\n",
+ "print('**Note down the last of these ten numbers, you will need to submit it for your coursework**')"
+ ],
+ "metadata": {
+ "id": "fNTBlg_GPEUh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Part (c)\n",
+ "\n",
+ "In part (c) of the coursework, you will calculate what proportion of the volume of a hypersphere is in the outer 1% of the radius/diameter. Calculate the volume of a hypersphere and then the volume of a hypersphere with 0.99 of the radius and then figure out the proportion (a number between 0 and 1). You'll see that by the time we get to 300 dimensions most of the volume is in the outer 1 percent. Extremely weird!"
+ ],
+ "metadata": {
+ "id": "GdyMeOBmoXyF"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def get_prop_of_volume_in_outer_1_percent(dimension):\n",
+ " # TODO -- replace this line\n",
+ " proportion = 1.0\n",
+ "\n",
+ " return proportion"
+ ],
+ "metadata": {
+ "id": "8_CxZ2AIpQ8w"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# While we're here, let's look at how much of the volume is in the outer 1% of the radius\n",
+ "for c_dim in [1,2,10,20,50,100,150,200,250,300]:\n",
+ " print('Proportion of volume in outer 1 percent of radius in %d dimensions =%3.3f'%(c_dim, get_prop_of_volume_in_outer_1_percent(c_dim)))\n",
+ "print('**Note down the last of these ten numbers, you will need to submit it for your coursework**')"
+ ],
+ "metadata": {
+ "id": "LtMDIn2qPVfJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/CM20315_2023/CM20315_Coursework_V_2023.ipynb b/CM20315_2023/CM20315_Coursework_V_2023.ipynb
new file mode 100644
index 0000000..483ea94
--- /dev/null
+++ b/CM20315_2023/CM20315_Coursework_V_2023.ipynb
@@ -0,0 +1,525 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyN7KaQQ63bf52r+b5as0MkK",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Coursework V: Backpropagation in Toy Model**\n",
+ "\n",
+ "This notebook computes the derivatives of a toy function similar (but different from) that in section 7.3 of the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions. At various points, you will get an answer that you need to copy into Moodle to be marked.\n",
+ "\n",
+ "Post to the content forum if you find any mistakes or need to clarify something."
+ ],
+ "metadata": {
+ "id": "pOZ6Djz0dhoy"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Problem setting\n",
+ "\n",
+ "We're going to investigate how to take the derivatives of functions where one operation is composed with another, which is composed with a third and so on. For example, consider the model:\n",
+ "\n",
+ "\\begin{equation}\n",
+ " \\mbox{f}[x,\\boldsymbol\\phi] = \\beta_3+\\omega_3\\cdot\\mbox{PReLU}\\Bigl[\\gamma, \\beta_2+\\omega_2\\cdot\\mbox{PReLU}\\bigl[\\gamma, \\beta_1+\\omega_1\\cdot\\mbox{PReLU}[\\gamma, \\beta_0+\\omega_0x]\\bigr]\\Bigr],\n",
+ "\\end{equation}\n",
+ "\n",
+ "with parameters $\\boldsymbol\\phi=\\{\\beta_0,\\omega_0,\\beta_1,\\omega_1,\\beta_2,\\omega_2,\\beta_3,\\omega_3\\}$, where\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\mbox{PReLU}[\\gamma, z] = \\begin{cases} \\gamma\\cdot z & \\quad z \\leq0 \\\\ z & \\quad z> 0\\end{cases}.\n",
+ "\\end{equation}\n",
+ "\n",
+ "Suppose that we have a binary cross-entropy loss function (equation 5.20 from the book):\n",
+ "\n",
+ "\\begin{equation*}\n",
+ "\\ell_i = -(1-y_{i})\\log\\Bigl[1-\\mbox{sig}[\\mbox{f}[\\mathbf{x}_i,\\boldsymbol\\phi]]\\Bigr] - y_{i}\\log\\Bigl[\\mbox{sig}[\\mbox{f}[\\mathbf{x}_i,\\boldsymbol\\phi]]\\Bigr].\n",
+ "\\end{equation*}\n",
+ "\n",
+ "Assume that we know the current values of $\\beta_{0},\\beta_{1},\\beta_{2},\\beta_{3},\\omega_{0},\\omega_{1},\\omega_{2},\\omega_{3}$, $\\gamma$, $x_i$ and $y_i$. We want to know how $\\ell_i$ changes when we make a small change to $\\beta_{0},\\beta_{1},\\beta_{2},\\beta_{3},\\omega_{0},\\omega_{1},\\omega_{2}$, or $\\omega_{3}$. In other words, we want to compute the eight derivatives:\n",
+ "\n",
+ "\\begin{eqnarray*}\n",
+ "\\frac{\\partial \\ell_i}{\\partial \\beta_{0}}, \\quad \\frac{\\partial \\ell_i}{\\partial \\beta_{1}}, \\quad \\frac{\\partial \\ell_i}{\\partial \\beta_{2}}, \\quad \\frac{\\partial \\ell_i }{\\partial \\beta_{3}}, \\quad \\frac{\\partial \\ell_i}{\\partial \\omega_{0}}, \\quad \\frac{\\partial \\ell_i}{\\partial \\omega_{1}}, \\quad \\frac{\\partial \\ell_i}{\\partial \\omega_{2}}, \\quad\\mbox{and} \\quad \\frac{\\partial \\ell_i}{\\partial \\omega_{3}}.\n",
+ "\\end{eqnarray*}"
+ ],
+ "metadata": {
+ "id": "1DmMo2w63CmT"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# import library\n",
+ "import numpy as np"
+ ],
+ "metadata": {
+ "id": "RIPaoVN834Lj"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's first define the original function and the loss term:"
+ ],
+ "metadata": {
+ "id": "32-ufWhc3v2c"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "AakK_qen3BpU"
+ },
+ "outputs": [],
+ "source": [
+ "# Defines the activation function\n",
+ "def paramReLU(gamma,x):\n",
+ " if x > 0:\n",
+ " return x\n",
+ " else:\n",
+ " return x * gamma\n",
+ "\n",
+ "# Defines the main function\n",
+ "def fn(x, beta0, beta1, beta2, beta3, omega0, omega1, omega2, omega3, gamma):\n",
+ " return beta3+omega3 * paramReLU(gamma, beta2 + omega2 * paramReLU(gamma, beta1 + omega1 * paramReLU(gamma, beta0 + omega0 * x)))\n",
+ "\n",
+ "# Logistic sigmoid\n",
+ "def sig(z):\n",
+ " return 1./(1+np.exp(-z))\n",
+ "\n",
+ "# The loss function (equation 5.20 from book)\n",
+ "def loss(f,y):\n",
+ " sig_net_out = sig(f)\n",
+ " l = -(1-y) * np.log(1-sig_net_out) - y * np.log(sig_net_out)\n",
+ " return l"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now we'll choose some values for the betas and the omegas and x and compute the output of the function:"
+ ],
+ "metadata": {
+ "id": "y7tf0ZMt5OXt"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "beta0 = 1.0; beta1 = -2.0; beta2 = -3.0; beta3 = 0.4\n",
+ "omega0 = 0.1; omega1 = -0.4; omega2 = 2.0; omega3 = -3.0\n",
+ "gamma = 0.2\n",
+ "x = 2.3; y =1.0\n",
+ "f_val = fn(x,beta0,beta1,beta2,beta3,omega0,omega1,omega2,omega3, gamma)\n",
+ "l_i_func = loss(f_val, y)\n",
+ "print('Loss full function = %3.3f'%l_i_func)"
+ ],
+ "metadata": {
+ "id": "pwvOcCxr41X_"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Forward pass\n",
+ "\n",
+ "We compute a series of intermediate values $f_0, h_0, f_1, h_1, f_2, h_2, f_3$, and finally the loss $\\ell$"
+ ],
+ "metadata": {
+ "id": "W6ZP62T5fU64"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "x = 2.3; y =1.0\n",
+ "gamma = 0.2\n",
+ "# Compute all the f_k and h_k terms\n",
+ "# I've done the first two for you\n",
+ "f0 = beta0+omega0 * x\n",
+ "h1 = paramReLU(gamma, f0)\n",
+ "\n",
+ "\n",
+ "# TODO: Replace the code below\n",
+ "f1 = 0\n",
+ "h2 = 0\n",
+ "f2 = 0\n",
+ "h3 = 0\n",
+ "f3 = 0\n",
+ "\n",
+ "\n",
+ "# Compute the loss and print\n",
+ "# The answer should be the same as when we computed the full function above\n",
+ "l_i = loss(f3, y)\n",
+ "print(\"Loss forward pass = %3.3f\"%(l_i))\n"
+ ],
+ "metadata": {
+ "id": "z-BckTpMf5PL"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Backward pass: Derivative of loss function with respect to function output\n",
+ "\n",
+ "Now, we'll compute the derivative $\\frac{dl}{df_3}$ of the loss function with respect to the network output $f_3$. In other words, we are asking how does the loss change as we make a small change in the network output.\n",
+ "\n",
+ "Since the loss it itself a function of $\\mbox{sig}[f_3]$ we'll compute this using the chain rule:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\frac{dl}{df_3} = \\frac{d\\mbox{sig}[f_3]}{df_3}\\cdot \\frac{dl}{d\\mbox{sig}[f_3]}\n",
+ "\\end{equation}\n",
+ "\n",
+ "Your job is to compute the two quantities on the right hand side.\n"
+ ],
+ "metadata": {
+ "id": "TbFbxz64Xz4I"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Compute the derivative of the the loss with respect to the function output f_val\n",
+ "def dl_df(f_val,y):\n",
+ " # Compute sigmoid of network output\n",
+ " sig_f_val = sig(f_val)\n",
+ " # Compute the derivative of loss with respect to network output using chain rule\n",
+ " dl_df_val = dsig_df(f_val) * dl_dsigf(sig_f_val, y)\n",
+ " # Return the derivative\n",
+ " return dl_df_val"
+ ],
+ "metadata": {
+ "id": "ZWKAq6HC90qV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# MOODLE ANSWER # Notebook V 1a: Copy this code when you have finished it.\n",
+ "\n",
+ "# Compute the derivative of the logistic sigmoid function with respect to its input (as a closed form solution)\n",
+ "def dsig_df(f_val):\n",
+ " # TODO Write this function\n",
+ " # Replace this line:\n",
+ " return 1\n",
+ "\n",
+ "# Compute the derivative of the loss with respect to the logistic sigmoid (as a closed form solution)\n",
+ "def dl_dsigf(sig_f_val, y):\n",
+ " # TODO Write this function\n",
+ " # Replace this line:\n",
+ " return 1"
+ ],
+ "metadata": {
+ "id": "lIngYAgPq-5I"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's run that for some f_val, y. Check previous practicals to see how you can check whether your answer is correct."
+ ],
+ "metadata": {
+ "id": "Q-j-i8khXzbK"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "y = 0.0\n",
+ "dl_df3 = dl_df(f3,y)\n",
+ "print(\"Moodle Answer Notebook V 1b: dldh3=%3.3f\"%(dl_df3))\n",
+ "\n",
+ "y= 1.0\n",
+ "dl_df3 = dl_df(f3,y)\n",
+ "print(\"Moodle Answer Notebook V 1c: dldh3=%3.3f\"%(dl_df3))"
+ ],
+ "metadata": {
+ "id": "Z7Lb5BibY50H"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Backward pass: Derivative of activation function with respect to preactivations\n",
+ "\n",
+ "Write a function to compute the derivative $\\frac{\\partial h}{\\partial f}$ of the activation function (parametric ReLU) with respect to its input.\n"
+ ],
+ "metadata": {
+ "id": "BA7QaOzejzZw"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# MOODLE ANSWER Notebook V 2a: Copy this code when you have finished it.\n",
+ "\n",
+ "def dh_df(gamma, f_val):\n",
+ " # TODO: Write this function\n",
+ " # Replace this line:\n",
+ " return 1\n"
+ ],
+ "metadata": {
+ "id": "bBPfPg04j-Qw"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's run that for some values of f_val. Check previous practicals to see how you can check whether your answer is correct."
+ ],
+ "metadata": {
+ "id": "QRNCM0EGk9-w"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "f_val_test = 0.6\n",
+ "dh_df_val = dh_df(gamma, f_val_test)\n",
+ "print(\"Moodle Answer Notebook V 2b: dhdf=%3.3f\"%(dh_df_val))\n",
+ "\n",
+ "f_val_test = -0.4\n",
+ "dh_df_val = dh_df(gamma, f_val_test)\n",
+ "print(\"Moodle Answer Notebook V 2c: dhdf=%3.3f\"%(dh_df_val))"
+ ],
+ "metadata": {
+ "id": "bql8VZIGk8Wy"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ " # Backward pass: Compute the derivatives of $l_i$ with respect to the intermediate quantities but in reverse order:\n",
+ "\n",
+ "\\begin{eqnarray}\n",
+ "\\frac{\\partial \\ell_i}{\\partial h_3}, \\quad \\frac{\\partial \\ell_i}{\\partial f_2}, \\quad\n",
+ "\\frac{\\partial \\ell_i}{\\partial h_2}, \\quad \\frac{\\partial \\ell_i}{\\partial f_1}, \\quad\\frac{\\partial \\ell_i}{\\partial h_1}, \\quad\\mbox{and} \\quad \\frac{\\partial \\ell_i}{\\partial f_0}.\n",
+ "\\end{eqnarray}\n",
+ "\n",
+ "The first of these derivatives can be calculated using the chain rule:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\frac{\\partial \\ell_i}{\\partial h_{3}} =\\frac{\\partial f_{3}}{\\partial h_{3}} \\frac{\\partial \\ell_i}{\\partial f_{3}} .\n",
+ "\\end{equation}\n",
+ "\n",
+ "The left-hand side asks how $\\ell_i$ changes when $h_{3}$ changes. The right-hand side says we can decompose this into (i) how $\\ell_i$ changes when $f_{3}$ changes and how $f_{3}$ changes when $h_{3}$ changes. So you get a chain of events happening: $h_{3}$ changes $f_{3}$, which changes $\\ell_i$, and the derivatives represent the effects of this chain. Notice that we computed the first of these derivatives already. The second term is the derivative of $\\beta_{3} + \\omega_{3}h_{3}$ with respect to $h_3$ which is simply $\\omega_3$. \n",
+ "\n",
+ "We can continue in this way, computing the derivatives of the output with respect to these intermediate quantities:\n",
+ "\n",
+ "\\begin{eqnarray}\n",
+ "\\frac{\\partial \\ell_i}{\\partial f_{2}} &=& \\frac{\\partial h_{3}}{\\partial f_{2}}\\left(\n",
+ "\\frac{\\partial f_{3}}{\\partial h_{3}}\\frac{\\partial \\ell_i}{\\partial f_{3}} \\right)\n",
+ "\\nonumber \\\\\n",
+ "\\frac{\\partial \\ell_i}{\\partial h_{2}} &=& \\frac{\\partial f_{2}}{\\partial h_{2}}\\left(\\frac{\\partial h_{3}}{\\partial f_{2}}\\frac{\\partial f_{3}}{\\partial h_{3}}\\frac{\\partial \\ell_i}{\\partial f_{3}}\\right)\\nonumber \\\\\n",
+ "\\frac{\\partial \\ell_i}{\\partial f_{1}} &=& \\frac{\\partial h_{2}}{\\partial f_{1}}\\left( \\frac{\\partial f_{2}}{\\partial h_{2}}\\frac{\\partial h_{3}}{\\partial f_{2}}\\frac{\\partial f_{3}}{\\partial h_{3}}\\frac{\\partial \\ell_i}{\\partial f_{3}} \\right)\\nonumber \\\\\n",
+ "\\frac{\\partial \\ell_i}{\\partial h_{1}} &=& \\frac{\\partial f_{1}}{\\partial h_{1}}\\left(\\frac{\\partial h_{2}}{\\partial f_{1}} \\frac{\\partial f_{2}}{\\partial h_{2}}\\frac{\\partial h_{3}}{\\partial f_{2}}\\frac{\\partial f_{3}}{\\partial h_{3}}\\frac{\\partial \\ell_i}{\\partial f_{3}} \\right)\\nonumber \\\\\n",
+ "\\frac{\\partial \\ell_i}{\\partial f_{0}} &=& \\frac{\\partial h_{1}}{\\partial f_{0}}\\left(\\frac{\\partial f_{1}}{\\partial h_{1}}\\frac{\\partial h_{2}}{\\partial f_{1}} \\frac{\\partial f_{2}}{\\partial h_{2}}\\frac{\\partial h_{3}}{\\partial f_{2}}\\frac{\\partial f_{3}}{\\partial h_{3}}\\frac{\\partial \\ell_i}{\\partial f_{3}} \\right).\n",
+ "\\end{eqnarray}\n",
+ "\n",
+ "In each case, we have already computed all of the terms except the last one in the previous step, and the last term is simple to evaluate. This is called the **backward pass**."
+ ],
+ "metadata": {
+ "id": "jay8NYWdFHuZ"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "x = 2.3; y =1.0\n",
+ "dldf3 = dl_df(f3,y)"
+ ],
+ "metadata": {
+ "id": "RSC_2CIfKF1b"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# MOODLE ANSWER Notebook V 3a: Copy this code when you have finished it.\n",
+ "# TODO -- Compute the derivatives of the output with respect\n",
+ "# to the intermediate computations h_k and f_k (i.e, run the backward pass)\n",
+ "# I've done the first two for you. You replace the code below:\n",
+ "# Replace the code below\n",
+ "dldh3 = 1\n",
+ "dldf2 = 1\n",
+ "dldh2 = 1\n",
+ "dldf1 = 1\n",
+ "dldh1 = 1\n",
+ "dldf0 = 1"
+ ],
+ "metadata": {
+ "id": "gCQJeI--Egdl"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Finally, we consider how the loss~$\\ell_{i}$ changes when we change the parameters $\\beta_{\\bullet}$ and $\\omega_{\\bullet}$. Once more, we apply the chain rule:\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\\begin{eqnarray}\n",
+ "\\frac{\\partial \\ell_i}{\\partial \\beta_{k}} &=& \\frac{\\partial f_{k}}{\\partial \\beta_{k}}\\frac{\\partial \\ell_i}{\\partial f_{k}}\\nonumber \\\\\n",
+ "\\frac{\\partial \\ell_i}{\\partial \\omega_{k}} &=& \\frac{\\partial f_{k}}{\\partial \\omega_{k}}\\frac{\\partial \\ell_i}{\\partial f_{k}}.\n",
+ "\\end{eqnarray}\n",
+ "\n",
+ "\\noindent In each case, the second term on the right-hand side was computed in step 2. When $k>0$, we have~$f_{k}=\\beta_{k}+\\omega_k \\cdot h_{k}$, so:\n",
+ "\n",
+ "\\begin{eqnarray}\n",
+ "\\frac{\\partial f_{k}}{\\partial \\beta_{k}} = 1 \\quad\\quad\\mbox{and}\\quad \\quad \\frac{\\partial f_{k}}{\\partial \\omega_{k}} &=& h_{k}.\n",
+ "\\end{eqnarray}"
+ ],
+ "metadata": {
+ "id": "FlzlThQPGpkU"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# MOODLE ANSWER Notebook V 3b: Copy this code when you have finished it.\n",
+ "# TODO -- Calculate the final derivatives with respect to the beta and omega terms\n",
+ "# Replace these terms\n",
+ "dldbeta3 = 1\n",
+ "dldomega3 = 1\n",
+ "dldbeta2 = 1\n",
+ "dldomega2 = 1\n",
+ "dldbeta1 = 1\n",
+ "dldomega1 = 1\n",
+ "dldbeta0 = 1\n",
+ "dldomega0 = 1"
+ ],
+ "metadata": {
+ "id": "1I2BhqZhGMK6"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Print the last two values out (enter these into Moodle). Again, think about how you can test whether these are correct.\n",
+ "print('Moodle Answer Notebook V 3c: dldbeta0=%3.3f'%(dldbeta0))\n",
+ "print('Moodle Answer Notebook V 3d: dldOmega0=%3.3f'%(dldomega0))"
+ ],
+ "metadata": {
+ "id": "38eiOn2aHgHI"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Compute the derivatives of $\\ell_i$ with respect to the parmeter $\\gamma$ of the parametric ReLU function. \n",
+ "\n",
+ "In other words, compute:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\frac{d\\ell_i}{d\\gamma}\n",
+ "\\end{equation}\n",
+ "\n",
+ "Along the way, we will need to compute derivatives\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\frac{dh_k(\\gamma,f_{k-1})}{d\\gamma}\n",
+ "\\end{equation}\n",
+ "\n",
+ "This is quite difficult and not worth many marks, so don't spend too much time on it if you are confused!"
+ ],
+ "metadata": {
+ "id": "lhD5AoUHx3DS"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Computes how an activation changes with a small change in gamma assuming preactivations are f\n",
+ "# MOODLE ANSWER # Notebook V 4a: Copy this code when you have finished it.\n",
+ "def dhdgamma(gamma, f):\n",
+ " # TODO -- Write this function\n",
+ " # Replace this line\n",
+ " return 1"
+ ],
+ "metadata": {
+ "id": "yC-9MTQevliP"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Compute how the loss changes with gamma\n",
+ "# Replace this line:\n",
+ "# MOODLE ANSWER # Notebook V 4b: Copy this code when you have finished it.\n",
+ "dldgamma = 1"
+ ],
+ "metadata": {
+ "id": "DiNQrveoLuHR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "print(\"Moodle Answer Notebook V 4c: dldgamma = %3.3f\"%(dldgamma))"
+ ],
+ "metadata": {
+ "id": "YHxmAEnxzy3O"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/LICENSE.txt b/LICENSE.txt
new file mode 100644
index 0000000..f335487
--- /dev/null
+++ b/LICENSE.txt
@@ -0,0 +1,346 @@
+Creative Commons Attribution-NonCommercial-NoDerivatives 4.0
+International Public License
+
+By exercising the Licensed Rights (defined below), You accept and agree
+to be bound by the terms and conditions of this Creative Commons
+Attribution-NonCommercial-NoDerivatives 4.0 International Public
+License ("Public License"). To the extent this Public License may be
+interpreted as a contract, You are granted the Licensed Rights in
+consideration of Your acceptance of these terms and conditions, and the
+Licensor grants You such rights in consideration of benefits the
+Licensor receives from making the Licensed Material available under
+these terms and conditions.
+
+
+Section 1 -- Definitions.
+
+ a. Adapted Material means material subject to Copyright and Similar
+ Rights that is derived from or based upon the Licensed Material
+ and in which the Licensed Material is translated, altered,
+ arranged, transformed, or otherwise modified in a manner requiring
+ permission under the Copyright and Similar Rights held by the
+ Licensor. For purposes of this Public License, where the Licensed
+ Material is a musical work, performance, or sound recording,
+ Adapted Material is always produced where the Licensed Material is
+ synched in timed relation with a moving image.
+
+ b. Copyright and Similar Rights means copyright and/or similar rights
+ closely related to copyright including, without limitation,
+ performance, broadcast, sound recording, and Sui Generis Database
+ Rights, without regard to how the rights are labeled or
+ categorized. For purposes of this Public License, the rights
+ specified in Section 2(b)(1)-(2) are not Copyright and Similar
+ Rights.
+
+ c. Effective Technological Measures means those measures that, in the
+ absence of proper authority, may not be circumvented under laws
+ fulfilling obligations under Article 11 of the WIPO Copyright
+ Treaty adopted on December 20, 1996, and/or similar international
+ agreements.
+
+ d. Exceptions and Limitations means fair use, fair dealing, and/or
+ any other exception or limitation to Copyright and Similar Rights
+ that applies to Your use of the Licensed Material.
+
+ e. Licensed Material means the artistic or literary work, database,
+ or other material to which the Licensor applied this Public
+ License.
+
+ f. Licensed Rights means the rights granted to You subject to the
+ terms and conditions of this Public License, which are limited to
+ all Copyright and Similar Rights that apply to Your use of the
+ Licensed Material and that the Licensor has authority to license.
+
+ g. Licensor means the individual(s) or entity(ies) granting rights
+ under this Public License.
+
+ h. NonCommercial means not primarily intended for or directed towards
+ commercial advantage or monetary compensation. For purposes of
+ this Public License, the exchange of the Licensed Material for
+ other material subject to Copyright and Similar Rights by digital
+ file-sharing or similar means is NonCommercial provided there is
+ no payment of monetary compensation in connection with the
+ exchange.
+
+ i. Share means to provide material to the public by any means or
+ process that requires permission under the Licensed Rights, such
+ as reproduction, public display, public performance, distribution,
+ dissemination, communication, or importation, and to make material
+ available to the public including in ways that members of the
+ public may access the material from a place and at a time
+ individually chosen by them.
+
+ j. Sui Generis Database Rights means rights other than copyright
+ resulting from Directive 96/9/EC of the European Parliament and of
+ the Council of 11 March 1996 on the legal protection of databases,
+ as amended and/or succeeded, as well as other essentially
+ equivalent rights anywhere in the world.
+
+ k. You means the individual or entity exercising the Licensed Rights
+ under this Public License. Your has a corresponding meaning.
+
+
+Section 2 -- Scope.
+
+ a. License grant.
+
+ 1. Subject to the terms and conditions of this Public License,
+ the Licensor hereby grants You a worldwide, royalty-free,
+ non-sublicensable, non-exclusive, irrevocable license to
+ exercise the Licensed Rights in the Licensed Material to:
+
+ a. reproduce and Share the Licensed Material, in whole or
+ in part, for NonCommercial purposes only; and
+
+ b. produce and reproduce, but not Share, Adapted Material
+ for NonCommercial purposes only.
+
+ 2. Exceptions and Limitations. For the avoidance of doubt, where
+ Exceptions and Limitations apply to Your use, this Public
+ License does not apply, and You do not need to comply with
+ its terms and conditions.
+
+ 3. Term. The term of this Public License is specified in Section
+ 6(a).
+
+ 4. Media and formats; technical modifications allowed. The
+ Licensor authorizes You to exercise the Licensed Rights in
+ all media and formats whether now known or hereafter created,
+ and to make technical modifications necessary to do so. The
+ Licensor waives and/or agrees not to assert any right or
+ authority to forbid You from making technical modifications
+ necessary to exercise the Licensed Rights, including
+ technical modifications necessary to circumvent Effective
+ Technological Measures. For purposes of this Public License,
+ simply making modifications authorized by this Section 2(a)
+ (4) never produces Adapted Material.
+
+ 5. Downstream recipients.
+
+ a. Offer from the Licensor -- Licensed Material. Every
+ recipient of the Licensed Material automatically
+ receives an offer from the Licensor to exercise the
+ Licensed Rights under the terms and conditions of this
+ Public License.
+
+ b. No downstream restrictions. You may not offer or impose
+ any additional or different terms or conditions on, or
+ apply any Effective Technological Measures to, the
+ Licensed Material if doing so restricts exercise of the
+ Licensed Rights by any recipient of the Licensed
+ Material.
+
+ 6. No endorsement. Nothing in this Public License constitutes or
+ may be construed as permission to assert or imply that You
+ are, or that Your use of the Licensed Material is, connected
+ with, or sponsored, endorsed, or granted official status by,
+ the Licensor or others designated to receive attribution as
+ provided in Section 3(a)(1)(A)(i).
+
+ b. Other rights.
+
+ 1. Moral rights, such as the right of integrity, are not
+ licensed under this Public License, nor are publicity,
+ privacy, and/or other similar personality rights; however, to
+ the extent possible, the Licensor waives and/or agrees not to
+ assert any such rights held by the Licensor to the limited
+ extent necessary to allow You to exercise the Licensed
+ Rights, but not otherwise.
+
+ 2. Patent and trademark rights are not licensed under this
+ Public License.
+
+ 3. To the extent possible, the Licensor waives any right to
+ collect royalties from You for the exercise of the Licensed
+ Rights, whether directly or through a collecting society
+ under any voluntary or waivable statutory or compulsory
+ licensing scheme. In all other cases the Licensor expressly
+ reserves any right to collect such royalties, including when
+ the Licensed Material is used other than for NonCommercial
+ purposes.
+
+
+Section 3 -- License Conditions.
+
+Your exercise of the Licensed Rights is expressly made subject to the
+following conditions.
+
+ a. Attribution.
+
+ 1. If You Share the Licensed Material, You must:
+
+ a. retain the following if it is supplied by the Licensor
+ with the Licensed Material:
+
+ i. identification of the creator(s) of the Licensed
+ Material and any others designated to receive
+ attribution, in any reasonable manner requested by
+ the Licensor (including by pseudonym if
+ designated);
+
+ ii. a copyright notice;
+
+ iii. a notice that refers to this Public License;
+
+ iv. a notice that refers to the disclaimer of
+ warranties;
+
+ v. a URI or hyperlink to the Licensed Material to the
+ extent reasonably practicable;
+
+ b. indicate if You modified the Licensed Material and
+ retain an indication of any previous modifications; and
+
+ c. indicate the Licensed Material is licensed under this
+ Public License, and include the text of, or the URI or
+ hyperlink to, this Public License.
+
+ For the avoidance of doubt, You do not have permission under
+ this Public License to Share Adapted Material.
+
+ 2. You may satisfy the conditions in Section 3(a)(1) in any
+ reasonable manner based on the medium, means, and context in
+ which You Share the Licensed Material. For example, it may be
+ reasonable to satisfy the conditions by providing a URI or
+ hyperlink to a resource that includes the required
+ information.
+
+ 3. If requested by the Licensor, You must remove any of the
+ information required by Section 3(a)(1)(A) to the extent
+ reasonably practicable.
+
+
+Section 4 -- Sui Generis Database Rights.
+
+Where the Licensed Rights include Sui Generis Database Rights that
+apply to Your use of the Licensed Material:
+
+ a. for the avoidance of doubt, Section 2(a)(1) grants You the right
+ to extract, reuse, reproduce, and Share all or a substantial
+ portion of the contents of the database for NonCommercial purposes
+ only and provided You do not Share Adapted Material;
+
+ b. if You include all or a substantial portion of the database
+ contents in a database in which You have Sui Generis Database
+ Rights, then the database in which You have Sui Generis Database
+ Rights (but not its individual contents) is Adapted Material; and
+
+ c. You must comply with the conditions in Section 3(a) if You Share
+ all or a substantial portion of the contents of the database.
+
+For the avoidance of doubt, this Section 4 supplements and does not
+replace Your obligations under this Public License where the Licensed
+Rights include other Copyright and Similar Rights.
+
+
+Section 5 -- Disclaimer of Warranties and Limitation of Liability.
+
+ a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE
+ EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS
+ AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF
+ ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS,
+ IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION,
+ WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR
+ PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS,
+ ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT
+ KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT
+ ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU.
+
+ b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE
+ TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION,
+ NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT,
+ INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES,
+ COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR
+ USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN
+ ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR
+ DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR
+ IN PART, THIS LIMITATION MAY NOT APPLY TO YOU.
+
+ c. The disclaimer of warranties and limitation of liability provided
+ above shall be interpreted in a manner that, to the extent
+ possible, most closely approximates an absolute disclaimer and
+ waiver of all liability.
+
+
+Section 6 -- Term and Termination.
+
+ a. This Public License applies for the term of the Copyright and
+ Similar Rights licensed here. However, if You fail to comply with
+ this Public License, then Your rights under this Public License
+ terminate automatically.
+
+ b. Where Your right to use the Licensed Material has terminated under
+ Section 6(a), it reinstates:
+
+ 1. automatically as of the date the violation is cured, provided
+ it is cured within 30 days of Your discovery of the
+ violation; or
+
+ 2. upon express reinstatement by the Licensor.
+
+ For the avoidance of doubt, this Section 6(b) does not affect any
+ right the Licensor may have to seek remedies for Your violations
+ of this Public License.
+
+ c. For the avoidance of doubt, the Licensor may also offer the
+ Licensed Material under separate terms or conditions or stop
+ distributing the Licensed Material at any time; however, doing so
+ will not terminate this Public License.
+
+ d. Sections 1, 5, 6, 7, and 8 survive termination of this Public
+ License.
+
+
+Section 7 -- Other Terms and Conditions.
+
+ a. The Licensor shall not be bound by any additional or different
+ terms or conditions communicated by You unless expressly agreed.
+
+ b. Any arrangements, understandings, or agreements regarding the
+ Licensed Material not stated herein are separate from and
+ independent of the terms and conditions of this Public License.
+
+
+Section 8 -- Interpretation.
+
+ a. For the avoidance of doubt, this Public License does not, and
+ shall not be interpreted to, reduce, limit, restrict, or impose
+ conditions on any use of the Licensed Material that could lawfully
+ be made without permission under this Public License.
+
+ b. To the extent possible, if any provision of this Public License is
+ deemed unenforceable, it shall be automatically reformed to the
+ minimum extent necessary to make it enforceable. If the provision
+ cannot be reformed, it shall be severed from this Public License
+ without affecting the enforceability of the remaining terms and
+ conditions.
+
+ c. No term or condition of this Public License will be waived and no
+ failure to comply consented to unless expressly agreed to by the
+ Licensor.
+
+ d. Nothing in this Public License constitutes or may be interpreted
+ as a limitation upon, or waiver of, any privileges and immunities
+ that apply to the Licensor or You, including from the legal
+ processes of any jurisdiction or authority.
+
+=======================================================================
+
+Creative Commons is not a party to its public
+licenses. Notwithstanding, Creative Commons may elect to apply one of
+its public licenses to material it publishes and in those instances
+will be considered the “Licensor.” The text of the Creative Commons
+public licenses is dedicated to the public domain under the CC0 Public
+Domain Dedication. Except for the limited purpose of indicating that
+material is shared under a Creative Commons public license or as
+otherwise permitted by the Creative Commons policies published at
+creativecommons.org/policies, Creative Commons does not authorize the
+use of the trademark "Creative Commons" or any other trademark or logo
+of Creative Commons without its prior written consent including,
+without limitation, in connection with any unauthorized modifications
+to any of its public licenses or any other arrangements,
+understandings, or agreements concerning use of licensed material. For
+the avoidance of doubt, this paragraph does not form part of the
+public licenses.
+
+Creative Commons may be contacted at creativecommons.org.
diff --git a/Notebooks/Chap01/1_1_BackgroundMathematics.ipynb b/Notebooks/Chap01/1_1_BackgroundMathematics.ipynb
new file mode 100644
index 0000000..c2fe012
--- /dev/null
+++ b/Notebooks/Chap01/1_1_BackgroundMathematics.ipynb
@@ -0,0 +1,423 @@
+{
+ "cells": [
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "colab_type": "text",
+ "id": "view-in-github"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "s5zzKSOusPOB"
+ },
+ "source": [
+ "\n",
+ "# **Notebook 1.1 -- Background Mathematics**\n",
+ "\n",
+ "The purpose of this Python notebook is to make sure you can use CoLab and to familiarize yourself with some of the background mathematical concepts that you are going to need to understand deep learning. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Notebook 2.1 Supervised Learning\n",
+ "\n",
+ "The purpose of this notebook is to explore the linear regression model discussed in Chapter 2 of the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "sfB2oX2RNvuF"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "uoYl2Gn3Nr52"
+ },
+ "outputs": [],
+ "source": [
+ "# Math library\n",
+ "import numpy as np\n",
+ "# Plotting library\n",
+ "import matplotlib.pyplot as plt"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Create some input / output data\n",
+ "x = np.array([0.03, 0.19, 0.34, 0.46, 0.78, 0.81, 1.08, 1.18, 1.39, 1.60, 1.65, 1.90])\n",
+ "y = np.array([0.67, 0.85, 1.05, 1.0, 1.40, 1.5, 1.3, 1.54, 1.55, 1.68, 1.73, 1.6 ])\n",
+ "\n",
+ "print(x)\n",
+ "print(y)"
+ ],
+ "metadata": {
+ "id": "MUbTD4znORtd"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define 1D linear regression model\n",
+ "def f(x, phi0, phi1):\n",
+ " # TODO : Replace this line with the linear regression model (eq 2.4)\n",
+ " y = x\n",
+ "\n",
+ " return y"
+ ],
+ "metadata": {
+ "id": "lw2dCRHwSW9a"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Function to help plot the data\n",
+ "def plot(x, y, phi0, phi1):\n",
+ " fig,ax = plt.subplots()\n",
+ " ax.scatter(x,y)\n",
+ " plt.xlim([0,2.0])\n",
+ " plt.ylim([0,2.0])\n",
+ " ax.set_xlabel('Input, $x$')\n",
+ " ax.set_ylabel('Output, $y$')\n",
+ " # Draw line\n",
+ " x_line = np.arange(0,2,0.01)\n",
+ " y_line = f(x_line, phi0, phi1)\n",
+ " plt.plot(x_line, y_line,'b-',lw=2)\n",
+ "\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "VT4F3xxSOt8C"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set the intercept and slope as in figure 2.2b\n",
+ "phi0 = 0.4 ; phi1 = 0.2\n",
+ "# Plot the data and the model\n",
+ "plot(x,y,phi0,phi1)"
+ ],
+ "metadata": {
+ "id": "AkdZdmhHWuVR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Function to calculate the loss\n",
+ "def compute_loss(x,y,phi0,phi1):\n",
+ "\n",
+ " # TODO Replace this line with the loss calculation (equation 2.5)\n",
+ " loss = 0\n",
+ "\n",
+ "\n",
+ " return loss"
+ ],
+ "metadata": {
+ "id": "1-GW218wX44b"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Compute the loss for our current model\n",
+ "loss = compute_loss(x,y,phi0,phi1)\n",
+ "print(f'Your Loss = {loss:3.2f}, Ground truth =7.07')"
+ ],
+ "metadata": {
+ "id": "Hgw7_GzBZ8tX"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set the intercept and slope as in figure 2.2c\n",
+ "phi0 = 1.60 ; phi1 =-0.8\n",
+ "# Plot the data and the model\n",
+ "plot(x,y,phi0,phi1)\n",
+ "loss = compute_loss(x,y,phi0,phi1)\n",
+ "print(f'Your Loss = {loss:3.2f}, Ground truth =10.28')"
+ ],
+ "metadata": {
+ "id": "_vZS28-FahGP"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TO DO -- Change the parameters manually to fit the model\n",
+ "# First fix phi1 and try changing phi0 until you can't make the loss go down any more\n",
+ "# Then fix phi0 and try changing phi1 until you can't make the loss go down any more\n",
+ "# Repeat this process until you find a set of parameters that fit the model as in figure 2.2d\n",
+ "# You can either do this by hand, or if you want to get fancy, write code to descent automatically in this way\n",
+ "# Start at these values:\n",
+ "phi0 = 1.60 ; phi1 =-0.8\n",
+ "\n",
+ "plot(x,y,phi0,phi1)\n",
+ "print(f'Your Loss = {compute_loss(x,y,phi0,phi1):3.2f}')"
+ ],
+ "metadata": {
+ "id": "VzpnzdW5d9vj"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Visualizing the loss function\n",
+ "\n",
+ "The above process is equivalent to to descending coordinate wise on the loss function"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "1Z6LB4Ybn1oN"
+ },
+ "source": [
+ "# **Notebook 3.1 -- Shallow neural networks I**\n",
+ "\n",
+ "The purpose of this notebook is to gain some familiarity with shallow neural networks with 1D inputs. It works through an example similar to figure 3.3 and experiments with different activation functions. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 3.2 -- Shallow neural networks II**\n",
+ "\n",
+ "The purpose of this notebook is to gain some familiarity with shallow neural networks with 2D inputs. It works through an example similar to figure 3.8 and experiments with different activation functions. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 3.3 -- Shallow network regions**\n",
+ "\n",
+ "The purpose of this notebook is to compute the maximum possible number of linear regions as seen in figure 3.9 of the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "DCTC8fQ6cp-n"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Imports math library\n",
+ "import numpy as np\n",
+ "# Imports plotting library\n",
+ "import matplotlib.pyplot as plt\n",
+ "# Imports math library\n",
+ "import math"
+ ],
+ "metadata": {
+ "id": "W3C1ZA1gcpq_"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The number of regions $N$ created by a shallow neural network with $D_i$ inputs and $D$ hidden units is given by Zaslavsky's formula:\n",
+ "\n",
+ "\\begin{equation}N = \\sum_{j=0}^{D_{i}}\\binom{D}{j}=\\sum_{j=0}^{D_{i}} \\frac{D!}{(D-j)!j!} \\end{equation} "
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Mn0F56yY8ohX"
+ },
+ "source": [
+ "# **Notebook 3.4 -- Activation functions**\n",
+ "\n",
+ "The purpose of this practical is to experiment with different activation functions. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "#Notebook 4.1 -- Composing networks\n",
+ "\n",
+ "The purpose of this notebook is to understand what happens when we feed one neural network into another. It works through an example similar to 4.1 and varies both networks\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions"
+ ],
+ "metadata": {
+ "id": "MaKn8CFlzN8E"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "8ClURpZQzI6L"
+ },
+ "outputs": [],
+ "source": [
+ "# Imports math library\n",
+ "import numpy as np\n",
+ "# Imports plotting library\n",
+ "import matplotlib.pyplot as plt"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the Rectified Linear Unit (ReLU) function\n",
+ "def ReLU(preactivation):\n",
+ " activation = preactivation.clip(0.0)\n",
+ " return activation"
+ ],
+ "metadata": {
+ "id": "YdmveeAUz4YG"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define a shallow neural network with, one input, one output, and three hidden units\n",
+ "def shallow_1_1_3(x, activation_fn, phi_0,phi_1,phi_2,phi_3, theta_10, theta_11, theta_20, theta_21, theta_30, theta_31):\n",
+ " # Initial lines\n",
+ " pre_1 = theta_10 + theta_11 * x\n",
+ " pre_2 = theta_20 + theta_21 * x\n",
+ " pre_3 = theta_30 + theta_31 * x\n",
+ " # Activation functions\n",
+ " act_1 = activation_fn(pre_1)\n",
+ " act_2 = activation_fn(pre_2)\n",
+ " act_3 = activation_fn(pre_3)\n",
+ " # Weight activations\n",
+ " w_act_1 = phi_1 * act_1\n",
+ " w_act_2 = phi_2 * act_2\n",
+ " w_act_3 = phi_3 * act_3\n",
+ " # Combine weighted activation and add y offset\n",
+ " y = phi_0 + w_act_1 + w_act_2 + w_act_3\n",
+ " # Return everything we have calculated\n",
+ " return y"
+ ],
+ "metadata": {
+ "id": "ximCLwIfz8kj"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# # Plot two shallow neural networks and the composition of the two\n",
+ "def plot_neural_two_components(x_in, net1_out, net2_out, net12_out=None):\n",
+ "\n",
+ " # Plot the two networks separately\n",
+ " fig, ax = plt.subplots(1,2)\n",
+ " fig.set_size_inches(8.5, 8.5)\n",
+ " fig.tight_layout(pad=3.0)\n",
+ " ax[0].plot(x_in, net1_out,'r-')\n",
+ " ax[0].set_xlabel('Net 1 input'); ax[0].set_ylabel('Net 1 output')\n",
+ " ax[0].set_xlim([-1,1]);ax[0].set_ylim([-1,1])\n",
+ " ax[0].set_aspect(1.0)\n",
+ " ax[1].plot(x_in, net2_out,'b-')\n",
+ " ax[1].set_xlabel('Net 2 input'); ax[1].set_ylabel('Net 2 output')\n",
+ " ax[1].set_xlim([-1,1]);ax[1].set_ylim([-1,1])\n",
+ " ax[1].set_aspect(1.0)\n",
+ " plt.show()\n",
+ "\n",
+ " if net12_out is not None:\n",
+ " # Plot their composition\n",
+ " fig, ax = plt.subplots()\n",
+ " ax.plot(x_in ,net12_out,'g-')\n",
+ " ax.set_xlabel('Net 1 Input'); ax.set_ylabel('Net 2 Output')\n",
+ " ax.set_xlim([-1,1]);ax.set_ylim([-1,1])\n",
+ " ax.set_aspect(1.0)\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "ZB2HTalOE40X"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's define two networks. We'll put the prefixes n1_ and n2_ before all the variables to make it clear which network is which. We'll just consider the inputs and outputs over the range [-1,1]."
+ ],
+ "metadata": {
+ "id": "LxBJCObC-NTY"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now lets define some parameters and run the first neural network\n",
+ "n1_theta_10 = 0.0 ; n1_theta_11 = -1.0\n",
+ "n1_theta_20 = 0 ; n1_theta_21 = 1.0\n",
+ "n1_theta_30 = -0.67 ; n1_theta_31 = 1.0\n",
+ "n1_phi_0 = 1.0; n1_phi_1 = -2.0; n1_phi_2 = -3.0; n1_phi_3 = 9.3\n",
+ "\n",
+ "# Now lets define some parameters and run the second neural network\n",
+ "n2_theta_10 = -0.6 ; n2_theta_11 = -1.0\n",
+ "n2_theta_20 = 0.2 ; n2_theta_21 = 1.0\n",
+ "n2_theta_30 = -0.5 ; n2_theta_31 = 1.0\n",
+ "n2_phi_0 = 0.5; n2_phi_1 = -1.0; n2_phi_2 = -1.5; n2_phi_3 = 2.0\n",
+ "\n",
+ "# Display the two inputs\n",
+ "x = np.arange(-1,1,0.001)\n",
+ "# We run the first and second neural networks for each of these input values\n",
+ "net1_out = shallow_1_1_3(x, ReLU, n1_phi_0, n1_phi_1, n1_phi_2, n1_phi_3, n1_theta_10, n1_theta_11, n1_theta_20, n1_theta_21, n1_theta_30, n1_theta_31)\n",
+ "net2_out = shallow_1_1_3(x, ReLU, n2_phi_0, n2_phi_1, n2_phi_2, n2_phi_3, n2_theta_10, n2_theta_11, n2_theta_20, n2_theta_21, n2_theta_30, n2_theta_31)\n",
+ "# Plot both graphs\n",
+ "plot_neural_two_components(x, net1_out, net2_out)"
+ ],
+ "metadata": {
+ "id": "JRebvurv22pT"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO\n",
+ "# Take a piece of paper and draw what you think will happen when we feed the\n",
+ "# output of the first network into the second one. Draw the relationship between\n",
+ "# the input of the first network and the output of the second one."
+ ],
+ "metadata": {
+ "id": "NUQVop9-Xta1"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now let's see if your predictions were right\n",
+ "\n",
+ "# TODO feed the output of first network into second network (replace this line)\n",
+ "net12_out = np.zeros_like(x)\n",
+ "\n",
+ "# Plot all three graphs\n",
+ "plot_neural_two_components(x, net1_out, net2_out, net12_out)"
+ ],
+ "metadata": {
+ "id": "Yq7GH-MCIyPI"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now we'll change things a up a bit. What happens if we change the second network? (note the *-1 change)\n",
+ "net1_out = shallow_1_1_3(x, ReLU, n1_phi_0, n1_phi_1, n1_phi_2, n1_phi_3, n1_theta_10, n1_theta_11, n1_theta_20, n1_theta_21, n1_theta_30, n1_theta_31)\n",
+ "net2_out = shallow_1_1_3(x, ReLU, n2_phi_0, n2_phi_1*-1, n2_phi_2, n2_phi_3, n2_theta_10, n2_theta_11, n2_theta_20, n2_theta_21, n2_theta_30, n2_theta_31)\n",
+ "plot_neural_two_components(x, net1_out, net2_out)"
+ ],
+ "metadata": {
+ "id": "BMlLkLbdEuPu"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO\n",
+ "# Take a piece of paper and draw what you think will happen when we feed the\n",
+ "# output of the first network into the modified second network. Draw the relationship between\n",
+ "# the input of the first network and the output of the second one."
+ ],
+ "metadata": {
+ "id": "Of6jVXLTJ688"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# When you have a prediction, run this code to see if you were right\n",
+ "net12_out = shallow_1_1_3(net1_out, ReLU, n2_phi_0, n2_phi_1*-1, n2_phi_2, n2_phi_3, n2_theta_10, n2_theta_11, n2_theta_20, n2_theta_21, n2_theta_30, n2_theta_31)\n",
+ "plot_neural_two_components(x, net1_out, net2_out, net12_out)"
+ ],
+ "metadata": {
+ "id": "PbbSCaSeK6SM"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's change things again. What happens if we change the first network? (note the changes)\n",
+ "net1_out = shallow_1_1_3(x, ReLU, n1_phi_0, n1_phi_1*0.5, n1_phi_2, n1_phi_3, n1_theta_10, n1_theta_11, n1_theta_20, n1_theta_21, n1_theta_30, n1_theta_31)\n",
+ "net2_out = shallow_1_1_3(x, ReLU, n2_phi_0, n2_phi_1, n2_phi_2, n2_phi_3, n2_theta_10, n2_theta_11, n2_theta_20, n2_theta_21, n2_theta_30, n2_theta_31)\n",
+ "plot_neural_two_components(x, net1_out, net2_out)"
+ ],
+ "metadata": {
+ "id": "b39mcSGFK9Fd"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO\n",
+ "# Take a piece of paper and draw what you think will happen when we feed the\n",
+ "# output of the modified first network into the original second network. Draw the relationship between\n",
+ "# the input of the first network and the output of the second one."
+ ],
+ "metadata": {
+ "id": "MhO40cC_LW9I"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# When you have a prediction, run this code to see if you were right\n",
+ "net12_out = shallow_1_1_3(net1_out, ReLU, n2_phi_0, n2_phi_1, n2_phi_2, n2_phi_3, n2_theta_10, n2_theta_11, n2_theta_20, n2_theta_21, n2_theta_30, n2_theta_31)\n",
+ "plot_neural_two_components(x, net1_out, net2_out, net12_out)"
+ ],
+ "metadata": {
+ "id": "Akwo-hnPLkNr"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's change things again. What happens if the first network and second networks are the same?\n",
+ "net1_out = shallow_1_1_3(x, ReLU, n1_phi_0, n1_phi_1, n1_phi_2, n1_phi_3, n1_theta_10, n1_theta_11, n1_theta_20, n1_theta_21, n1_theta_30, n1_theta_31)\n",
+ "net2_out_new = shallow_1_1_3(x, ReLU, n1_phi_0, n1_phi_1, n1_phi_2, n1_phi_3, n1_theta_10, n1_theta_11, n1_theta_20, n1_theta_21, n1_theta_30, n1_theta_31)\n",
+ "plot_neural_two_components(x, net1_out, net2_out_new)"
+ ],
+ "metadata": {
+ "id": "TJ7wXKpRLl_E"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO\n",
+ "# Take a piece of paper and draw what you think will happen when we feed the\n",
+ "# output of the first network into the a copy of itself. Draw the relationship between\n",
+ "# the input of the first network and the output of the second one."
+ ],
+ "metadata": {
+ "id": "dJbbh6R7NG9k"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# When you have a prediction, run this code to see if you were right\n",
+ "net12_out = shallow_1_1_3(net1_out, ReLU, n1_phi_0, n1_phi_1, n1_phi_2, n1_phi_3, n1_theta_10, n1_theta_11, n1_theta_20, n1_theta_21, n1_theta_30, n1_theta_31)\n",
+ "plot_neural_two_components(x, net1_out, net2_out_new, net12_out)"
+ ],
+ "metadata": {
+ "id": "BiZZl3yNM2Bq"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO\n",
+ "# Contemplate what you think will happen when we feed the\n",
+ "# output of the original first network into a second copy of the original first network, and then\n",
+ "# the output of that into the original second network (so now we have a three layer network)\n",
+ "# How many total linear regions will we have in the output?\n",
+ "net123_out = shallow_1_1_3(net12_out, ReLU, n2_phi_0, n2_phi_1, n2_phi_2, n2_phi_3, n2_theta_10, n2_theta_11, n2_theta_20, n2_theta_21, n2_theta_30, n2_theta_31)\n",
+ "plot_neural_two_components(x, net12_out, net2_out, net123_out)"
+ ],
+ "metadata": {
+ "id": "BSd51AkzNf7-"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TO DO\n",
+ "# How many linear regions would there be if we ran N copies of the first network, feeding the result of the first\n",
+ "# into the second, the second into the third and so on, and then passed the result into the original second\n",
+ "# network (blue curve above)\n",
+ "\n",
+ "# Take away conclusion: with very few parameters, we can make A LOT of linear regions, but\n",
+ "# they depend on one another in complex ways that quickly become too difficult to understand intuitively."
+ ],
+ "metadata": {
+ "id": "HqzePCLOVQK7"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap04/4_2_Clipping_functions.ipynb b/Notebooks/Chap04/4_2_Clipping_functions.ipynb
new file mode 100644
index 0000000..82c725b
--- /dev/null
+++ b/Notebooks/Chap04/4_2_Clipping_functions.ipynb
@@ -0,0 +1,219 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyPkFrjmRAUf0fxN07RC4xMI",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "#Notebook 4.2 -- Clipping functions\n",
+ "\n",
+ "The purpose of this notebook is to understand how a neural network with two hidden layers build more complicated functions by clipping and recombining the representations at the intermediate hidden variables.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions"
+ ],
+ "metadata": {
+ "id": "MaKn8CFlzN8E"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "8ClURpZQzI6L"
+ },
+ "outputs": [],
+ "source": [
+ "# Imports math library\n",
+ "import numpy as np\n",
+ "# Imports plotting library\n",
+ "import matplotlib.pyplot as plt"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the Rectified Linear Unit (ReLU) function\n",
+ "def ReLU(preactivation):\n",
+ " activation = preactivation.clip(0.0)\n",
+ " return activation"
+ ],
+ "metadata": {
+ "id": "YdmveeAUz4YG"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define a deep neural network with, one input, one output, two hidden layers and three hidden units (eqns 4.7-4.9)\n",
+ "# To make this easier, we store the parameters in ndarrays, so phi_0 = phi[0] and psi_3,3 = psi[3,3] etc.\n",
+ "def shallow_1_1_3_3(x, activation_fn, phi, psi, theta):\n",
+ "\n",
+ " # TODO -- You write this function\n",
+ " # Replace the skeleton code below.\n",
+ "\n",
+ " # ANSWER\n",
+ " # Preactivations at layer 1 (terms in brackets in equation 4.7)\n",
+ " layer1_pre_1 = np.zeros_like(x) ;\n",
+ " layer1_pre_2 = np.zeros_like(x) ;\n",
+ " layer1_pre_3 = np.zeros_like(x) ;\n",
+ "\n",
+ " # Activation functions (rest of equation 4.7)\n",
+ " h1 = activation_fn(layer1_pre_1)\n",
+ " h2 = activation_fn(layer1_pre_2)\n",
+ " h3 = activation_fn(layer1_pre_3)\n",
+ "\n",
+ " # Preactivations at layer 2 (terms in brackets in equation 4.8)\n",
+ " layer2_pre_1 = np.zeros_like(x) ;\n",
+ " layer2_pre_2 = np.zeros_like(x) ;\n",
+ " layer2_pre_3 = np.zeros_like(x) ;\n",
+ "\n",
+ " # Activation functions (rest of equation 4.8)\n",
+ " h1_prime = activation_fn(layer2_pre_1)\n",
+ " h2_prime = activation_fn(layer2_pre_2)\n",
+ " h3_prime = activation_fn(layer2_pre_3)\n",
+ "\n",
+ " # Weighted outputs by phi (three last terms of equation 4.9)\n",
+ " phi1_h1_prime = np.zeros_like(x) ;\n",
+ " phi2_h2_prime = np.zeros_like(x) ;\n",
+ " phi3_h3_prime = np.zeros_like(x) ;\n",
+ "\n",
+ " # Combine weighted activation and add y offset (summing terms of equation 4.9)\n",
+ " y = np.zeros_like(x) ;\n",
+ "\n",
+ "\n",
+ " # Return everything we have calculated\n",
+ " return y, layer2_pre_1, layer2_pre_2, layer2_pre_3, h1_prime, h2_prime, h3_prime, phi1_h1_prime, phi2_h2_prime, phi3_h3_prime"
+ ],
+ "metadata": {
+ "id": "ximCLwIfz8kj"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# # Plot two layer neural network as in figure 4.5\n",
+ "def plot_neural_two_layers(x, y, layer2_pre_1, layer2_pre_2, layer2_pre_3, h1_prime, h2_prime, h3_prime, phi1_h1_prime, phi2_h2_prime, phi3_h3_prime):\n",
+ "\n",
+ " fig, ax = plt.subplots(3,3)\n",
+ " fig.set_size_inches(8.5, 8.5)\n",
+ " fig.tight_layout(pad=3.0)\n",
+ " ax[0,0].plot(x,layer2_pre_1,'r-'); ax[0,0].set_ylabel('$\\psi_{10}+\\psi_{11}h_{1}+\\psi_{12}h_{2}+\\psi_{13}h_3$')\n",
+ " ax[0,1].plot(x,layer2_pre_2,'b-'); ax[0,1].set_ylabel('$\\psi_{20}+\\psi_{21}h_{1}+\\psi_{22}h_{2}+\\psi_{23}h_3$')\n",
+ " ax[0,2].plot(x,layer2_pre_3,'g-'); ax[0,2].set_ylabel('$\\psi_{30}+\\psi_{31}h_{1}+\\psi_{32}h_{2}+\\psi_{33}h_3$')\n",
+ " ax[1,0].plot(x,h1_prime,'r-'); ax[1,0].set_ylabel(\"$h_{1}^{'}$\")\n",
+ " ax[1,1].plot(x,h2_prime,'b-'); ax[1,1].set_ylabel(\"$h_{2}^{'}$\")\n",
+ " ax[1,2].plot(x,h3_prime,'g-'); ax[1,2].set_ylabel(\"$h_{3}^{'}$\")\n",
+ " ax[2,0].plot(x,phi1_h1_prime,'r-'); ax[2,0].set_ylabel(\"$\\phi_1 h_{1}^{'}$\")\n",
+ " ax[2,1].plot(x,phi2_h2_prime,'b-'); ax[2,1].set_ylabel(\"$\\phi_2 h_{2}^{'}$\")\n",
+ " ax[2,2].plot(x,phi3_h3_prime,'g-'); ax[2,2].set_ylabel(\"$\\phi_3 h_{3}^{'}$\")\n",
+ "\n",
+ " for plot_y in range(3):\n",
+ " for plot_x in range(3):\n",
+ " ax[plot_y,plot_x].set_xlim([0,1]);ax[plot_x,plot_y].set_ylim([-1,1])\n",
+ " ax[plot_y,plot_x].set_aspect(0.5)\n",
+ " ax[2,plot_y].set_xlabel('Input, $x$');\n",
+ " plt.show()\n",
+ "\n",
+ " fig, ax = plt.subplots()\n",
+ " ax.plot(x,y)\n",
+ " ax.set_xlabel('Input, $x$'); ax.set_ylabel('Output, $y$')\n",
+ " ax.set_xlim([0,1]);ax.set_ylim([-1,1])\n",
+ " ax.set_aspect(0.5)\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "ZB2HTalOE40X"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's define the parameters and visualize the network"
+ ],
+ "metadata": {
+ "id": "LxBJCObC-NTY"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define parameters (note first dimension of theta and phi is padded to make indices match\n",
+ "# notation in book)\n",
+ "theta = np.zeros([4,2])\n",
+ "psi = np.zeros([4,4])\n",
+ "phi = np.zeros([4,1])\n",
+ "\n",
+ "theta[1,0] = 0.3 ; theta[1,1] = -1.0\n",
+ "theta[2,0]= -1.0 ; theta[2,1] = 2.0\n",
+ "theta[3,0] = -0.5 ; theta[3,1] = 0.65\n",
+ "psi[1,0] = 0.3; psi[1,1] = 2.0; psi[1,2] = -1.0; psi[1,3]=7.0\n",
+ "psi[2,0] = -0.2; psi[2,1] = 2.0; psi[2,2] = 1.2; psi[2,3]=-8.0\n",
+ "psi[3,0] = 0.3; psi[3,1] = -2.3; psi[3,2] = -0.8; psi[3,3]=2.0\n",
+ "phi[0] = 0.0; phi[1] = 0.5; phi[2] = -1.5; phi [3] = 2.2\n",
+ "\n",
+ "# Define a range of input values\n",
+ "x = np.arange(0,1,0.01)\n",
+ "\n",
+ "# Run the neural network\n",
+ "y, layer2_pre_1, layer2_pre_2, layer2_pre_3, h1_prime, h2_prime, h3_prime, phi1_h1_prime, phi2_h2_prime, phi3_h3_prime \\\n",
+ " = shallow_1_1_3_3(x, ReLU, phi, psi, theta)\n",
+ "\n",
+ "# And then plot it\n",
+ "plot_neural_two_layers(x, y, layer2_pre_1, layer2_pre_2, layer2_pre_3, h1_prime, h2_prime, h3_prime, phi1_h1_prime, phi2_h2_prime, phi3_h3_prime)"
+ ],
+ "metadata": {
+ "id": "JRebvurv22pT"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "To do: To test your understanding of this, consider:\n",
+ "\n",
+ "1. What would happen if we increase $\\psi_{1,0}$?\n",
+ "2. What would happen if we multiplied $\\psi_{2,0}, \\psi_{2,1}, \\psi_{2,2}, \\psi_{2,3}$ by -1?\n",
+ "3. What would happen if set $\\phi_{3}$ to -1?\n",
+ "\n",
+ "You can rerun the code to see if you were correct.\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "GcjUUHbXf25D"
+ }
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap04/4_3_Deep_Networks.ipynb b/Notebooks/Chap04/4_3_Deep_Networks.ipynb
new file mode 100644
index 0000000..5754e95
--- /dev/null
+++ b/Notebooks/Chap04/4_3_Deep_Networks.ipynb
@@ -0,0 +1,321 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyO2DaD75p+LGi7WgvTzjrk1",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 4.3 Deep neural networks**\n",
+ "\n",
+ "This network investigates converting neural networks to matrix form.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "MaKn8CFlzN8E"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "8ClURpZQzI6L"
+ },
+ "outputs": [],
+ "source": [
+ "# Imports math library\n",
+ "import numpy as np\n",
+ "# Imports plotting library\n",
+ "import matplotlib.pyplot as plt"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the Rectified Linear Unit (ReLU) function\n",
+ "def ReLU(preactivation):\n",
+ " activation = preactivation.clip(0.0)\n",
+ " return activation"
+ ],
+ "metadata": {
+ "id": "YdmveeAUz4YG"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define a shallow neural network with, one input, one output, and three hidden units\n",
+ "def shallow_1_1_3(x, activation_fn, phi_0,phi_1,phi_2,phi_3, theta_10, theta_11, theta_20, theta_21, theta_30, theta_31):\n",
+ " # Initial lines\n",
+ " pre_1 = theta_10 + theta_11 * x\n",
+ " pre_2 = theta_20 + theta_21 * x\n",
+ " pre_3 = theta_30 + theta_31 * x\n",
+ " # Activation functions\n",
+ " act_1 = activation_fn(pre_1)\n",
+ " act_2 = activation_fn(pre_2)\n",
+ " act_3 = activation_fn(pre_3)\n",
+ " # Weight activations\n",
+ " w_act_1 = phi_1 * act_1\n",
+ " w_act_2 = phi_2 * act_2\n",
+ " w_act_3 = phi_3 * act_3\n",
+ " # Combine weighted activation and add y offset\n",
+ " y = phi_0 + w_act_1 + w_act_2 + w_act_3\n",
+ " # Return everything we have calculated\n",
+ " return y, pre_1, pre_2, pre_3, act_1, act_2, act_3, w_act_1, w_act_2, w_act_3"
+ ],
+ "metadata": {
+ "id": "ximCLwIfz8kj"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# # Plot the shallow neural network. We'll assume input in is range [-1,1] and output [-1,1]\n",
+ "def plot_neural(x, y):\n",
+ " fig, ax = plt.subplots()\n",
+ " ax.plot(x.T,y.T)\n",
+ " ax.set_xlabel('Input'); ax.set_ylabel('Output')\n",
+ " ax.set_xlim([-1,1]);ax.set_ylim([-1,1])\n",
+ " ax.set_aspect(1.0)\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "btrt7BX20gKD"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's define a network. We'll just consider the inputs and outputs over the range [-1,1]."
+ ],
+ "metadata": {
+ "id": "LxBJCObC-NTY"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now lets define some parameters and run the first neural network\n",
+ "n1_theta_10 = 0.0 ; n1_theta_11 = -1.0\n",
+ "n1_theta_20 = 0 ; n1_theta_21 = 1.0\n",
+ "n1_theta_30 = -0.67 ; n1_theta_31 = 1.0\n",
+ "n1_phi_0 = 1.0; n1_phi_1 = -2.0; n1_phi_2 = -3.0; n1_phi_3 = 9.3\n",
+ "\n",
+ "# Define a range of input values\n",
+ "n1_in = np.arange(-1,1,0.01).reshape([1,-1])\n",
+ "\n",
+ "# We run the neural network for each of these input values\n",
+ "n1_out, *_ = shallow_1_1_3(n1_in, ReLU, n1_phi_0, n1_phi_1, n1_phi_2, n1_phi_3, n1_theta_10, n1_theta_11, n1_theta_20, n1_theta_21, n1_theta_30, n1_theta_31)\n",
+ "# And then plot it\n",
+ "plot_neural(n1_in, n1_out)"
+ ],
+ "metadata": {
+ "id": "JRebvurv22pT"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now we'll define the same neural network, but this time, we will use matrix form. When you get this right, it will draw the same plot as above."
+ ],
+ "metadata": {
+ "id": "XCJqo_AjfAra"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "beta_0 = np.zeros((3,1))\n",
+ "Omega_0 = np.zeros((3,1))\n",
+ "beta_1 = np.zeros((1,1))\n",
+ "Omega_1 = np.zeros((1,3))\n",
+ "\n",
+ "# TODO Fill in the values of the beta and Omega matrices with the n1_theta and n1_phi parameters that define the network above\n",
+ "# !!! NOTE THAT MATRICES ARE CONVENTIONALLY INDEXED WITH a_11 IN THE TOP LEFT CORNER, BUT NDARRAYS START AT [0,0]\n",
+ "# To get you started I've filled in a couple:\n",
+ "beta_0[0,0] = n1_theta_10\n",
+ "Omega_0[0,0] = n1_theta_11\n",
+ "\n",
+ "# Make sure that input data matrix has different inputs in its columns\n",
+ "n_data = n1_in.size\n",
+ "n_dim_in = 1\n",
+ "n1_in_mat = np.reshape(n1_in,(n_dim_in,n_data))\n",
+ "\n",
+ "# This runs the network for ALL of the inputs, x at once so we can draw graph\n",
+ "h1 = ReLU(np.matmul(beta_0,np.ones((1,n_data))) + np.matmul(Omega_0,n1_in_mat))\n",
+ "n1_out = np.matmul(beta_1,np.ones((1,n_data))) + np.matmul(Omega_1,h1)\n",
+ "\n",
+ "# Draw the network and check that it looks the same as the non-matrix case\n",
+ "plot_neural(n1_in, n1_out)"
+ ],
+ "metadata": {
+ "id": "MR0AecZYfACR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now we'll feed the output of the first network into the second one."
+ ],
+ "metadata": {
+ "id": "qOcj2Rof-o20"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now lets define some parameters and run the second neural network\n",
+ "n2_theta_10 = -0.6 ; n2_theta_11 = -1.0\n",
+ "n2_theta_20 = 0.2 ; n2_theta_21 = 1.0\n",
+ "n2_theta_30 = -0.5 ; n2_theta_31 = 1.0\n",
+ "n2_phi_0 = 0.5; n2_phi_1 = -1.0; n2_phi_2 = -1.5; n2_phi_3 = 2.0\n",
+ "\n",
+ "# Define a range of input values\n",
+ "n2_in = np.arange(-1,1,0.01)\n",
+ "\n",
+ "# We run the second neural network on the output of the first network\n",
+ "n2_out, *_ = \\\n",
+ " shallow_1_1_3(n1_out, ReLU, n2_phi_0, n2_phi_1, n2_phi_2, n2_phi_3, n2_theta_10, n2_theta_11, n2_theta_20, n2_theta_21, n2_theta_30, n2_theta_31)\n",
+ "# And then plot it\n",
+ "plot_neural(n1_in, n2_out)"
+ ],
+ "metadata": {
+ "id": "ZRjWu8i9239X"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "beta_0 = np.zeros((3,1))\n",
+ "Omega_0 = np.zeros((3,1))\n",
+ "beta_1 = np.zeros((3,1))\n",
+ "Omega_1 = np.zeros((3,3))\n",
+ "beta_2 = np.zeros((1,1))\n",
+ "Omega_2 = np.zeros((1,3))\n",
+ "\n",
+ "# TODO Fill in the values of the beta and Omega matrices for the n1_theta, n1_phi, n2_theta, and n2_phi parameters\n",
+ "# that define the composition of the two networks above (see eqn 4.5 for Omega1 and beta1 albeit in different notation)\n",
+ "# !!! NOTE THAT MATRICES ARE CONVENTIONALLY INDEXED WITH a_11 IN THE TOP LEFT CORNER, BUT NDARRAYS START AT [0,0] SO EVERYTHING IS OFFSET\n",
+ "# To get you started I've filled in a few:\n",
+ "beta_0[0,0] = n1_theta_10\n",
+ "Omega_0[0,0] = n1_theta_11\n",
+ "beta_1[0,0] = n2_theta_10 + n2_theta_11 * n1_phi_0\n",
+ "Omega_1[0,0] = n2_theta_11 * n1_phi_1\n",
+ "\n",
+ "\n",
+ "# Make sure that input data matrix has different inputs in its columns\n",
+ "n_data = n1_in.size\n",
+ "n_dim_in = 1\n",
+ "n1_in_mat = np.reshape(n1_in,(n_dim_in,n_data))\n",
+ "\n",
+ "# This runs the network for ALL of the inputs, x at once so we can draw graph (hence extra np.ones term)\n",
+ "h1 = ReLU(np.matmul(beta_0,np.ones((1,n_data))) + np.matmul(Omega_0,n1_in_mat))\n",
+ "h2 = ReLU(np.matmul(beta_1,np.ones((1,n_data))) + np.matmul(Omega_1,h1))\n",
+ "n1_out = np.matmul(beta_2,np.ones((1,n_data))) + np.matmul(Omega_2,h2)\n",
+ "\n",
+ "# Draw the network and check that it looks the same as the non-matrix version\n",
+ "plot_neural(n1_in, n1_out)"
+ ],
+ "metadata": {
+ "id": "ZB2HTalOE40X"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's make a deep network with 3 hidden layers. It will have $D_i=4$ inputs, $D_1=5$ neurons in the first layer, $D_2=2$ neurons in the second layer and $D_3=4$ neurons in the third layer, and $D_o = 1$ output. Consult figure 4.6 and equations 4.15 for guidance."
+ ],
+ "metadata": {
+ "id": "0VANqxH2kyS4"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# define sizes\n",
+ "D_i=4; D_1=5; D_2=2; D_3=4; D_o=1\n",
+ "# We'll choose the inputs and parameters of this network randomly using np.random.normal\n",
+ "# For example, we'll set the input using\n",
+ "n_data = 4;\n",
+ "x = np.random.normal(size=(D_i, n_data))\n",
+ "# TODO initialize the parameters randomly with the correct sizes\n",
+ "# Replace the lines below\n",
+ "beta_0 = np.random.normal(size=(1,1))\n",
+ "Omega_0 = np.random.normal(size=(1,1))\n",
+ "beta_1 = np.random.normal(size=(1,1))\n",
+ "Omega_1 = np.random.normal(size=(1,1))\n",
+ "beta_2 = np.random.normal(size=(1,1))\n",
+ "Omega_2 = np.random.normal(size=(1,1))\n",
+ "beta_3 = np.random.normal(size=(1,1))\n",
+ "Omega_3 = np.random.normal(size=(1,1))\n",
+ "\n",
+ "\n",
+ "# If you set the parameters to the correct sizes, the following code will run\n",
+ "h1 = ReLU(np.matmul(beta_0,np.ones((1,n_data))) + np.matmul(Omega_0,x));\n",
+ "h2 = ReLU(np.matmul(beta_1,np.ones((1,n_data))) + np.matmul(Omega_1,h1));\n",
+ "h3 = ReLU(np.matmul(beta_2,np.ones((1,n_data))) + np.matmul(Omega_2,h2));\n",
+ "y = np.matmul(beta_3,np.ones((1,n_data))) + np.matmul(Omega_3,h3)\n",
+ "\n",
+ "if h1.shape[0] is not D_1 or h1.shape[1] is not n_data:\n",
+ " print(\"h1 is wrong shape\")\n",
+ "if h2.shape[0] is not D_2 or h1.shape[1] is not n_data:\n",
+ " print(\"h2 is wrong shape\")\n",
+ "if h3.shape[0] is not D_3 or h1.shape[1] is not n_data:\n",
+ " print(\"h3 is wrong shape\")\n",
+ "if y.shape[0] is not D_o or h1.shape[1] is not n_data:\n",
+ " print(\"Output is wrong shape\")\n",
+ "\n",
+ "# Print the inputs and outputs\n",
+ "print(\"Input data points\")\n",
+ "print(x)\n",
+ "print (\"Output data points\")\n",
+ "print(y)"
+ ],
+ "metadata": {
+ "id": "RdBVAc_Rj22-"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
diff --git a/Notebooks/Chap05/5_1_Least_Squares_Loss.ipynb b/Notebooks/Chap05/5_1_Least_Squares_Loss.ipynb
new file mode 100644
index 0000000..5df709e
--- /dev/null
+++ b/Notebooks/Chap05/5_1_Least_Squares_Loss.ipynb
@@ -0,0 +1,593 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 5.1: Least Squares Loss**\n",
+ "\n",
+ "This notebook investigates the least squares loss and the equivalence of maximum likelihood and minimum negative log likelihood.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "jSlFkICHwHQF"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "PYMZ1x-Pv1ht"
+ },
+ "outputs": [],
+ "source": [
+ "# Imports math library\n",
+ "import numpy as np\n",
+ "# Imports plotting library\n",
+ "import matplotlib.pyplot as plt\n",
+ "# Import math Library\n",
+ "import math"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the Rectified Linear Unit (ReLU) function\n",
+ "def ReLU(preactivation):\n",
+ " activation = preactivation.clip(0.0)\n",
+ " return activation\n",
+ "\n",
+ "# Define a shallow neural network\n",
+ "def shallow_nn(x, beta_0, omega_0, beta_1, omega_1):\n",
+ " # Make sure that input data is (1 x n_data) array\n",
+ " n_data = x.size\n",
+ " x = np.reshape(x,(1,n_data))\n",
+ "\n",
+ " # This runs the network for ALL of the inputs, x at once so we can draw graph\n",
+ " h1 = ReLU(np.matmul(beta_0,np.ones((1,n_data))) + np.matmul(omega_0,x))\n",
+ " y = np.matmul(beta_1,np.ones((1,n_data))) + np.matmul(omega_1,h1)\n",
+ " return y"
+ ],
+ "metadata": {
+ "id": "Fv7SZR3tv7mV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Get parameters for model -- we can call this function to easily reset them\n",
+ "def get_parameters():\n",
+ " # And we'll create a network that approximately fits it\n",
+ " beta_0 = np.zeros((3,1)); # formerly theta_x0\n",
+ " omega_0 = np.zeros((3,1)); # formerly theta_x1\n",
+ " beta_1 = np.zeros((1,1)); # formerly phi_0\n",
+ " omega_1 = np.zeros((1,3)); # formerly phi_x\n",
+ "\n",
+ " beta_0[0,0] = 0.3; beta_0[1,0] = -1.0; beta_0[2,0] = -0.5\n",
+ " omega_0[0,0] = -1.0; omega_0[1,0] = 1.8; omega_0[2,0] = 0.65\n",
+ " beta_1[0,0] = 0.1;\n",
+ " omega_1[0,0] = -2.0; omega_1[0,1] = -1.0; omega_1[0,2] = 7.0\n",
+ "\n",
+ " return beta_0, omega_0, beta_1, omega_1"
+ ],
+ "metadata": {
+ "id": "pUT9Ain_HRim"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Utility function for plotting data\n",
+ "def plot_univariate_regression(x_model, y_model, x_data = None, y_data = None, sigma_model = None, title= None):\n",
+ " # Make sure model data are 1D arrays\n",
+ " x_model = np.squeeze(x_model)\n",
+ " y_model = np.squeeze(y_model)\n",
+ "\n",
+ " fig, ax = plt.subplots()\n",
+ " ax.plot(x_model,y_model)\n",
+ " if sigma_model is not None:\n",
+ " ax.fill_between(x_model, y_model-2*sigma_model, y_model+2*sigma_model, color='lightgray')\n",
+ " ax.set_xlabel('Input, $x$'); ax.set_ylabel('Output, $y$')\n",
+ " ax.set_xlim([0,1]);ax.set_ylim([-1,1])\n",
+ " ax.set_aspect(0.5)\n",
+ " if title is not None:\n",
+ " ax.set_title(title)\n",
+ " if x_data is not None:\n",
+ " ax.plot(x_data, y_data, 'ko')\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "NRR67ri_1TzN"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Univariate regression\n",
+ "\n",
+ "We'll investigate a simple univariate regression situation with a single input $x$ and a single output $y$ as pictured in figures 5.4 and 5.5b."
+ ],
+ "metadata": {
+ "id": "PsgLZwsPxauP"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's create some 1D training data\n",
+ "x_train = np.array([0.09291784,0.46809093,0.93089486,0.67612654,0.73441752,0.86847339,\\\n",
+ " 0.49873225,0.51083168,0.18343972,0.99380898,0.27840809,0.38028817,\\\n",
+ " 0.12055708,0.56715537,0.92005746,0.77072270,0.85278176,0.05315950,\\\n",
+ " 0.87168699,0.58858043])\n",
+ "y_train = np.array([-0.25934537,0.18195445,0.651270150,0.13921448,0.09366691,0.30567674,\\\n",
+ " 0.372291170,0.20716968,-0.08131792,0.51187806,0.16943738,0.3994327,\\\n",
+ " 0.019062570,0.55820410,0.452564960,-0.1183121,0.02957665,-1.24354444, \\\n",
+ " 0.248038840,0.26824970])\n",
+ "\n",
+ "# Get parameters for the model\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "sigma = 0.2\n",
+ "\n",
+ "# Define a range of input values\n",
+ "x_model = np.arange(0,1,0.01)\n",
+ "# Run the model to get values to plot and plot it.\n",
+ "y_model = shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
+ "plot_univariate_regression(x_model, y_model, x_train, y_train, sigma_model = sigma)\n"
+ ],
+ "metadata": {
+ "id": "VWzNOt1swFVd"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The blue line is the mean prediction of the model and the gray area represents plus/minus two standard deviations. This model fits okay, but could be improved. Let's compute the loss. We'll compute the the least squares error, the likelihood, the negative log likelihood."
+ ],
+ "metadata": {
+ "id": "MvVX6tl9AEXF"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Return probability under normal distribution\n",
+ "def normal_distribution(y, mu, sigma):\n",
+ " # TODO-- write in the equation for the normal distribution\n",
+ " # Equation 5.7 from the notes (you will need np.sqrt() and np.exp(), and math.pi)\n",
+ " # Don't use the numpy version -- that's cheating!\n",
+ " # Replace the line below\n",
+ " prob = np.zeros_like(y)\n",
+ "\n",
+ " return prob"
+ ],
+ "metadata": {
+ "id": "YaLdRlEX0FkU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's double check we get the right answer before proceeding\n",
+ "print(\"Correct answer = %3.3f, Your answer = %3.3f\"%(0.119,normal_distribution(1,-1,2.3)))"
+ ],
+ "metadata": {
+ "id": "4TSL14dqHHbV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's plot the Gaussian distribution.\n",
+ "y_gauss = np.arange(-5,5,0.1)\n",
+ "mu = 0; sigma = 1.0\n",
+ "gauss_prob = normal_distribution(y_gauss, mu, sigma)\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(y_gauss, gauss_prob)\n",
+ "ax.set_xlabel('Input, $y$'); ax.set_ylabel('Probability $Pr(y)$')\n",
+ "ax.set_xlim([-5,5]);ax.set_ylim([0,1.0])\n",
+ "plt.show()\n",
+ "\n",
+ "# TODO\n",
+ "# 1. Predict what will happen if we change to mu=1 and leave sigma=1\n",
+ "# Now change the code above and see if you were correct.\n",
+ "\n",
+ "# 2. Predict what will happen if we leave mu = 0 and change sigma to 2.0\n",
+ "\n",
+ "# 3. Predict what will happen if we leave mu = 0 and change sigma to 0.5"
+ ],
+ "metadata": {
+ "id": "A2HcmNfUMIlj"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's compute the likelihood using this function"
+ ],
+ "metadata": {
+ "id": "R5z_0dzQMF35"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Return the likelihood of all of the data under the model\n",
+ "def compute_likelihood(y_train, mu, sigma):\n",
+ " # TODO -- compute the likelihood of the data -- the product of the normal probabilities for each data point\n",
+ " # Top line of equation 5.3 in the notes\n",
+ " # You will need np.prod() and the normal_distribution function you used above\n",
+ " # Replace the line below\n",
+ " likelihood = 0\n",
+ "\n",
+ " return likelihood"
+ ],
+ "metadata": {
+ "id": "zpS7o6liCx7f"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's test this for a homoscedastic (constant sigma) model\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "# Use our neural network to predict the mean of the Gaussian\n",
+ "mu_pred = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+ "# Set the standard deviation to something reasonable\n",
+ "sigma = 0.2\n",
+ "# Compute the likelihood\n",
+ "likelihood = compute_likelihood(y_train, mu_pred, sigma)\n",
+ "# Let's double check we get the right answer before proceeding\n",
+ "print(\"Correct answer = %9.9f, Your answer = %9.9f\"%(0.000010624,likelihood))"
+ ],
+ "metadata": {
+ "id": "1hQxBLoVNlr2"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "You can see that this gives a very small answer, even for this small 1D dataset, and with the model fitting quite well. This is because it is the product of several probabilities, which are all quite small themselves.\n",
+ "This will get out of hand pretty quickly with real datasets -- the likelihood will get so small that we can't represent it with normal finite-precision math\n",
+ "\n",
+ "This is why we use negative log likelihood"
+ ],
+ "metadata": {
+ "id": "HzphKgPfOvlk"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Return the negative log likelihood of the data under the model\n",
+ "def compute_negative_log_likelihood(y_train, mu, sigma):\n",
+ " # TODO -- compute the negative log likelihood of the data without using a product\n",
+ " # In other words, compute minus one times the sum of the log probabilities\n",
+ " # Equation 5.4 in the notes\n",
+ " # You will need np.sum(), np.log()\n",
+ " # Replace the line below\n",
+ " nll = 0\n",
+ "\n",
+ " return nll"
+ ],
+ "metadata": {
+ "id": "dsT0CWiKBmTV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's test this for a homoscedastic (constant sigma) model\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "# Use our neural network to predict the mean of the Gaussian\n",
+ "mu_pred = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+ "# Set the standard deviation to something reasonable\n",
+ "sigma = 0.2\n",
+ "# Compute the negative log likelihood\n",
+ "nll = compute_negative_log_likelihood(y_train, mu_pred, sigma)\n",
+ "# Let's double check we get the right answer before proceeding\n",
+ "print(\"Correct answer = %9.9f, Your answer = %9.9f\"%(11.452419564,nll))"
+ ],
+ "metadata": {
+ "id": "nVxUXg9rQmwI"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "For good measure, let's compute the sum of squares as well"
+ ],
+ "metadata": {
+ "id": "-S8bXApoWVLG"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Return the squared distance between the observed data (y_train) and the prediction of the model (y_pred)\n",
+ "def compute_sum_of_squares(y_train, y_pred):\n",
+ " # TODO -- compute the sum of squared distances between the training data and the model prediction\n",
+ " # Eqn 5.10 in the notes. Make sure that you understand this, and ask questions if you don't\n",
+ " # Replace the line below\n",
+ " sum_of_squares = 0;\n",
+ "\n",
+ " return sum_of_squares"
+ ],
+ "metadata": {
+ "id": "I1pjFdHCF4JZ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's test this again\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "# Use our neural network to predict the mean of the Gaussian, which is out best prediction of y\n",
+ "y_pred = mu_pred = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+ "# Compute the sum of squares\n",
+ "sum_of_squares = compute_sum_of_squares(y_train, y_pred)\n",
+ "# Let's double check we get the right answer before proceeding\n",
+ "print(\"Correct answer = %9.9f, Your answer = %9.9f\"%(2.020992572,sum_of_squares))"
+ ],
+ "metadata": {
+ "id": "2C40fskIHBx7"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's investigate finding the maximum likelihood / minimum negative log likelihood / least squares solution. For simplicity, we'll assume that all the parameters are correct except one and look at how the likelihood, negative log likelihood, and sum of squares change as we manipulate the last parameter. We'll start with overall y offset, beta_1 (formerly phi_0)"
+ ],
+ "metadata": {
+ "id": "OgcRojvPWh4V"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define a range of values for the parameter\n",
+ "beta_1_vals = np.arange(0,1.0,0.01)\n",
+ "# Create some arrays to store the likelihoods, negative log likelihoods and sum of squares\n",
+ "likelihoods = np.zeros_like(beta_1_vals)\n",
+ "nlls = np.zeros_like(beta_1_vals)\n",
+ "sum_squares = np.zeros_like(beta_1_vals)\n",
+ "\n",
+ "# Initialise the parameters\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "sigma = 0.2\n",
+ "for count in range(len(beta_1_vals)):\n",
+ " # Set the value for the parameter\n",
+ " beta_1[0,0] = beta_1_vals[count]\n",
+ " # Run the network with new parameters\n",
+ " mu_pred = y_pred = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+ " # Compute and store the three values\n",
+ " likelihoods[count] = compute_likelihood(y_train, mu_pred, sigma)\n",
+ " nlls[count] = compute_negative_log_likelihood(y_train, mu_pred, sigma)\n",
+ " sum_squares[count] = compute_sum_of_squares(y_train, y_pred)\n",
+ " # Draw the model for every 20th parameter setting\n",
+ " if count % 20 == 0:\n",
+ " # Run the model to get values to plot and plot it.\n",
+ " y_model = shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
+ " plot_univariate_regression(x_model, y_model, x_train, y_train, sigma_model = sigma, title=\"beta1=%3.3f\"%(beta_1[0,0]))\n"
+ ],
+ "metadata": {
+ "id": "pFKtDaAeVU4U"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now let's plot the likelihood, negative log likelihood, and least squares as a function of the value of the offset beta1\n",
+ "fig, ax = plt.subplots(1,2)\n",
+ "fig.set_size_inches(10.5, 5.5)\n",
+ "fig.tight_layout(pad=10.0)\n",
+ "likelihood_color = 'tab:red'\n",
+ "nll_color = 'tab:blue'\n",
+ "\n",
+ "ax[0].set_xlabel('beta_1[0]')\n",
+ "ax[0].set_ylabel('likelihood', color = likelihood_color)\n",
+ "ax[0].plot(beta_1_vals, likelihoods, color = likelihood_color)\n",
+ "ax[0].tick_params(axis='y', labelcolor=likelihood_color)\n",
+ "\n",
+ "ax00 = ax[0].twinx()\n",
+ "ax00.plot(beta_1_vals, nlls, color = nll_color)\n",
+ "ax00.set_ylabel('negative log likelihood', color = nll_color)\n",
+ "ax00.tick_params(axis='y', labelcolor = nll_color)\n",
+ "\n",
+ "plt.axvline(x = beta_1_vals[np.argmax(likelihoods)], linestyle='dotted')\n",
+ "\n",
+ "ax[1].plot(beta_1_vals, sum_squares); ax[1].set_xlabel('beta_1[0]'); ax[1].set_ylabel('sum of squares')\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "UHXeTa9MagO6"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Hopefully, you can see that the maximum of the likelihood fn is at the same position as the minimum negative log likelihood\n",
+ "# and the least squares solutions\n",
+ "# Let's check that:\n",
+ "print(\"Maximum likelihood = %3.3f, at beta_1=%3.3f\"%( (likelihoods[np.argmax(likelihoods)],beta_1_vals[np.argmax(likelihoods)])))\n",
+ "print(\"Minimum negative log likelihood = %3.3f, at beta_1=%3.3f\"%( (nlls[np.argmin(nlls)],beta_1_vals[np.argmin(nlls)])))\n",
+ "print(\"Least squares = %3.3f, at beta_1=%3.3f\"%( (sum_squares[np.argmin(sum_squares)],beta_1_vals[np.argmin(sum_squares)])))\n",
+ "\n",
+ "# Plot the best model\n",
+ "beta_1[0,0] = beta_1_vals[np.argmin(sum_squares)]\n",
+ "y_model = shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
+ "plot_univariate_regression(x_model, y_model, x_train, y_train, sigma_model = sigma, title=\"beta1=%3.3f\"%(beta_1[0,0]))"
+ ],
+ "metadata": {
+ "id": "aDEPhddNdN4u"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "They all give the same answer. But you can see from the three plots above that the likelihood is very small unless the parameters are almost correct. So in practice, we would work with the negative log likelihood or the least squares."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 5.2 Binary Cross-Entropy Loss**\n",
+ "\n",
+ "This notebook investigates the binary cross-entropy loss. It follows from applying the formula in section 5.2 to a loss function based on the Bernoulli distribution.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "jSlFkICHwHQF"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "PYMZ1x-Pv1ht"
+ },
+ "outputs": [],
+ "source": [
+ "# Imports math library\n",
+ "import numpy as np\n",
+ "# Imports plotting library\n",
+ "import matplotlib.pyplot as plt\n",
+ "# Import math Library\n",
+ "import math"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the Rectified Linear Unit (ReLU) function\n",
+ "def ReLU(preactivation):\n",
+ " activation = preactivation.clip(0.0)\n",
+ " return activation\n",
+ "\n",
+ "# Define a shallow neural network\n",
+ "def shallow_nn(x, beta_0, omega_0, beta_1, omega_1):\n",
+ " # Make sure that input data is (1 x n_data) array\n",
+ " n_data = x.size\n",
+ " x = np.reshape(x,(1,n_data))\n",
+ "\n",
+ " # This runs the network for ALL of the inputs, x at once so we can draw graph\n",
+ " h1 = ReLU(np.matmul(beta_0,np.ones((1,n_data))) + np.matmul(omega_0,x))\n",
+ " model_out = np.matmul(beta_1,np.ones((1,n_data))) + np.matmul(omega_1,h1)\n",
+ " return model_out"
+ ],
+ "metadata": {
+ "id": "Fv7SZR3tv7mV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Get parameters for model -- we can call this function to easily reset them\n",
+ "def get_parameters():\n",
+ " # And we'll create a network that approximately fits it\n",
+ " beta_0 = np.zeros((3,1)); # formerly theta_x0\n",
+ " omega_0 = np.zeros((3,1)); # formerly theta_x1\n",
+ " beta_1 = np.zeros((1,1)); # formerly phi_0\n",
+ " omega_1 = np.zeros((1,3)); # formerly phi_x\n",
+ "\n",
+ " beta_0[0,0] = 0.3; beta_0[1,0] = -1.0; beta_0[2,0] = -0.5\n",
+ " omega_0[0,0] = -1.0; omega_0[1,0] = 1.8; omega_0[2,0] = 0.65\n",
+ " beta_1[0,0] = 2.6;\n",
+ " omega_1[0,0] = -24.0; omega_1[0,1] = -8.0; omega_1[0,2] = 50.0\n",
+ "\n",
+ " return beta_0, omega_0, beta_1, omega_1"
+ ],
+ "metadata": {
+ "id": "pUT9Ain_HRim"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Utility function for plotting data\n",
+ "def plot_binary_classification(x_model, out_model, lambda_model, x_data = None, y_data = None, title= None):\n",
+ " # Make sure model data are 1D arrays\n",
+ " x_model = np.squeeze(x_model)\n",
+ " out_model = np.squeeze(out_model)\n",
+ " lambda_model = np.squeeze(lambda_model)\n",
+ "\n",
+ " fig, ax = plt.subplots(1,2)\n",
+ " fig.set_size_inches(7.0, 3.5)\n",
+ " fig.tight_layout(pad=3.0)\n",
+ " ax[0].plot(x_model,out_model)\n",
+ " ax[0].set_xlabel('Input, $x$'); ax[0].set_ylabel('Model output')\n",
+ " ax[0].set_xlim([0,1]);ax[0].set_ylim([-4,4])\n",
+ " if title is not None:\n",
+ " ax[0].set_title(title)\n",
+ " ax[1].plot(x_model,lambda_model)\n",
+ " ax[1].set_xlabel('Input, $x$'); ax[1].set_ylabel('$\\lambda$ or Pr(y=1|x)')\n",
+ " ax[1].set_xlim([0,1]);ax[1].set_ylim([-0.05,1.05])\n",
+ " if title is not None:\n",
+ " ax[1].set_title(title)\n",
+ " if x_data is not None:\n",
+ " ax[1].plot(x_data, y_data, 'ko')\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "NRR67ri_1TzN"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Binary classification\n",
+ "\n",
+ "In binary classification tasks, the network predicts the probability of the output belonging to class 1. Since probabilities must lie in [0,1] and the network can output arbitrary values, we map the network through a sigmoid function that ensures the range is valid."
+ ],
+ "metadata": {
+ "id": "PsgLZwsPxauP"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Sigmoid function that maps [-infty,infty] to [0,1]\n",
+ "def sigmoid(model_out):\n",
+ " # TODO -- implement the logistic sigmoid function from equation 5.18\n",
+ " # Replace this line:\n",
+ " sig_model_out = np.zeros_like(model_out)\n",
+ "\n",
+ " return sig_model_out"
+ ],
+ "metadata": {
+ "id": "uFb8h-9IXnIe"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's create some 1D training data\n",
+ "x_train = np.array([0.09291784,0.46809093,0.93089486,0.67612654,0.73441752,0.86847339,\\\n",
+ " 0.49873225,0.51083168,0.18343972,0.99380898,0.27840809,0.38028817,\\\n",
+ " 0.12055708,0.56715537,0.92005746,0.77072270,0.85278176,0.05315950,\\\n",
+ " 0.87168699,0.58858043])\n",
+ "y_train = np.array([0,1,1,0,0,1,\\\n",
+ " 1,0,0,1,0,1,\\\n",
+ " 0,1,1,0,1,0, \\\n",
+ " 1,1])\n",
+ "\n",
+ "# Get parameters for the model\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "\n",
+ "# Define a range of input values\n",
+ "x_model = np.arange(0,1,0.01)\n",
+ "# Run the model to get values to plot and plot it.\n",
+ "model_out= shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
+ "lambda_model = sigmoid(model_out)\n",
+ "plot_binary_classification(x_model, model_out, lambda_model, x_train, y_train)\n"
+ ],
+ "metadata": {
+ "id": "VWzNOt1swFVd"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The left is model output and the right is the model output after the sigmoid has been applied, so it now lies in the range [0,1] and represents the probability, that y=1. The black dots show the training data. We'll compute the likelihood and the negative log likelihood."
+ ],
+ "metadata": {
+ "id": "MvVX6tl9AEXF"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Return probability under Bernoulli distribution for observed class y\n",
+ "def bernoulli_distribution(y, lambda_param):\n",
+ " # TODO-- write in the equation for the Bernoulli distribution\n",
+ " # Equation 5.17 from the notes (you will need np.power)\n",
+ " # Replace the line below\n",
+ " prob = np.zeros_like(y)\n",
+ "\n",
+ " return prob"
+ ],
+ "metadata": {
+ "id": "YaLdRlEX0FkU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's double check we get the right answer before proceeding\n",
+ "print(\"Correct answer = %3.3f, Your answer = %3.3f\"%(0.8,bernoulli_distribution(0,0.2)))\n",
+ "print(\"Correct answer = %3.3f, Your answer = %3.3f\"%(0.2,bernoulli_distribution(1,0.2)))"
+ ],
+ "metadata": {
+ "id": "4TSL14dqHHbV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's compute the likelihood using this function"
+ ],
+ "metadata": {
+ "id": "R5z_0dzQMF35"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Return the likelihood of all of the data under the model\n",
+ "def compute_likelihood(y_train, lambda_param):\n",
+ " # TODO -- compute the likelihood of the data -- the product of the Bernoulli probabilities for each data point\n",
+ " # Top line of equation 5.3 in the notes\n",
+ " # You will need np.prod() and the bernoulli_distribution function you used above\n",
+ " # Replace the line below\n",
+ " likelihood = 0\n",
+ "\n",
+ " return likelihood"
+ ],
+ "metadata": {
+ "id": "zpS7o6liCx7f"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's test this\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "# Use our neural network to predict the Bernoulli parameter lambda\n",
+ "model_out = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+ "lambda_train = sigmoid(model_out)\n",
+ "# Compute the likelihood\n",
+ "likelihood = compute_likelihood(y_train, lambda_train)\n",
+ "# Let's double check we get the right answer before proceeding\n",
+ "print(\"Correct answer = %9.9f, Your answer = %9.9f\"%(0.000070237,likelihood))"
+ ],
+ "metadata": {
+ "id": "1hQxBLoVNlr2"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "You can see that this gives a very small answer, even for this small 1D dataset, and with the model fitting quite well. This is because it is the product of several probabilities, which are all quite small themselves.\n",
+ "This will get out of hand pretty quickly with real datasets -- the likelihood will get so small that we can't represent it with normal finite-precision math\n",
+ "\n",
+ "This is why we use negative log likelihood"
+ ],
+ "metadata": {
+ "id": "HzphKgPfOvlk"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Return the negative log likelihood of the data under the model\n",
+ "def compute_negative_log_likelihood(y_train, lambda_param):\n",
+ " # TODO -- compute the likelihood of the data -- don't use the likelihood function above -- compute the negative sum of the log probabilities\n",
+ " # You will need np.sum(), np.log()\n",
+ " # Replace the line below\n",
+ " nll = 0\n",
+ "\n",
+ " return nll"
+ ],
+ "metadata": {
+ "id": "dsT0CWiKBmTV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's test this\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "# Use our neural network to predict the mean of the Gaussian\n",
+ "model_out = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+ "# Pass through the sigmoid function\n",
+ "lambda_train = sigmoid(model_out)\n",
+ "# Compute the log likelihood\n",
+ "nll = compute_negative_log_likelihood(y_train, lambda_train)\n",
+ "# Let's double check we get the right answer before proceeding\n",
+ "print(\"Correct answer = %9.9f, Your answer = %9.9f\"%(9.563639387,nll))"
+ ],
+ "metadata": {
+ "id": "nVxUXg9rQmwI"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's investigate finding the maximum likelihood / minimum negative log likelihood solution. For simplicity, we'll assume that all the parameters are fixed except one and look at how the likelihood and negative log likelihood change as we manipulate the last parameter. We'll start with overall y_offset, beta_1 (formerly phi_0)"
+ ],
+ "metadata": {
+ "id": "OgcRojvPWh4V"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define a range of values for the parameter\n",
+ "beta_1_vals = np.arange(-2,6.0,0.1)\n",
+ "# Create some arrays to store the likelihoods, negative log likelihoods\n",
+ "likelihoods = np.zeros_like(beta_1_vals)\n",
+ "nlls = np.zeros_like(beta_1_vals)\n",
+ "\n",
+ "# Initialise the parameters\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "for count in range(len(beta_1_vals)):\n",
+ " # Set the value for the parameter\n",
+ " beta_1[0,0] = beta_1_vals[count]\n",
+ " # Run the network with new parameters\n",
+ " model_out = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+ " lambda_train = sigmoid(model_out)\n",
+ " # Compute and store the two values\n",
+ " likelihoods[count] = compute_likelihood(y_train,lambda_train)\n",
+ " nlls[count] = compute_negative_log_likelihood(y_train, lambda_train)\n",
+ " # Draw the model for every 20th parameter setting\n",
+ " if count % 20 == 0:\n",
+ " # Run the model to get values to plot and plot it.\n",
+ " model_out = shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
+ " lambda_model = sigmoid(model_out)\n",
+ " plot_binary_classification(x_model, model_out, lambda_model, x_train, y_train, title=\"beta_1[0]=%3.3f\"%(beta_1[0,0]))\n"
+ ],
+ "metadata": {
+ "id": "pFKtDaAeVU4U"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now let's plot the likelihood and negative log likelihood as a function of the value of the offset beta1\n",
+ "fig, ax = plt.subplots()\n",
+ "fig.tight_layout(pad=5.0)\n",
+ "likelihood_color = 'tab:red'\n",
+ "nll_color = 'tab:blue'\n",
+ "\n",
+ "\n",
+ "ax.set_xlabel('beta_1[0]')\n",
+ "ax.set_ylabel('likelihood', color = likelihood_color)\n",
+ "ax.plot(beta_1_vals, likelihoods, color = likelihood_color)\n",
+ "ax.tick_params(axis='y', labelcolor=likelihood_color)\n",
+ "\n",
+ "ax1 = ax.twinx()\n",
+ "ax1.plot(beta_1_vals, nlls, color = nll_color)\n",
+ "ax1.set_ylabel('negative log likelihood', color = nll_color)\n",
+ "ax1.tick_params(axis='y', labelcolor = nll_color)\n",
+ "\n",
+ "plt.axvline(x = beta_1_vals[np.argmax(likelihoods)], linestyle='dotted')\n",
+ "\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "UHXeTa9MagO6"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Hopefully, you can see that the maximum of the likelihood fn is at the same position as the minimum negative log likelihood\n",
+ "# Let's check that:\n",
+ "print(\"Maximum likelihood = %f, at beta_1=%3.3f\"%( (likelihoods[np.argmax(likelihoods)],beta_1_vals[np.argmax(likelihoods)])))\n",
+ "print(\"Minimum negative log likelihood = %f, at beta_1=%3.3f\"%( (nlls[np.argmin(nlls)],beta_1_vals[np.argmin(nlls)])))\n",
+ "\n",
+ "# Plot the best model\n",
+ "beta_1[0,0] = beta_1_vals[np.argmin(nlls)]\n",
+ "model_out = shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
+ "lambda_model = sigmoid(model_out)\n",
+ "plot_binary_classification(x_model, model_out, lambda_model, x_train, y_train, title=\"beta_1[0]=%3.3f\"%(beta_1[0,0]))\n"
+ ],
+ "metadata": {
+ "id": "aDEPhddNdN4u"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "They both give the same answer. But you can see from the likelihood above that the likelihood is very small unless the parameters are almost correct. So in practice, we would work with the negative log likelihood."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "jSlFkICHwHQF"
+ },
+ "source": [
+ "# **Notebook 5.3 Multiclass Cross-Entropy Loss**\n",
+ "\n",
+ "This notebook investigates the multi-class cross-entropy loss. It follows from applying the formula in section 5.2 to a loss function based on the Categorical distribution.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "PYMZ1x-Pv1ht"
+ },
+ "outputs": [],
+ "source": [
+ "# Imports math library\n",
+ "import numpy as np\n",
+ "# Used for repmat\n",
+ "import numpy.matlib\n",
+ "# Imports plotting library\n",
+ "import matplotlib.pyplot as plt\n",
+ "# Import math Library\n",
+ "import math"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Fv7SZR3tv7mV"
+ },
+ "outputs": [],
+ "source": [
+ "# Define the Rectified Linear Unit (ReLU) function\n",
+ "def ReLU(preactivation):\n",
+ " activation = preactivation.clip(0.0)\n",
+ " return activation\n",
+ "\n",
+ "# Define a shallow neural network\n",
+ "def shallow_nn(x, beta_0, omega_0, beta_1, omega_1):\n",
+ " # Make sure that input data is (1 x n_data) array\n",
+ " n_data = x.size\n",
+ " x = np.reshape(x,(1,n_data))\n",
+ "\n",
+ " # This runs the network for ALL of the inputs, x at once so we can draw graph\n",
+ " h1 = ReLU(np.matmul(beta_0,np.ones((1,n_data))) + np.matmul(omega_0,x))\n",
+ " model_out = np.matmul(beta_1,np.ones((1,n_data))) + np.matmul(omega_1,h1)\n",
+ " return model_out"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "pUT9Ain_HRim"
+ },
+ "outputs": [],
+ "source": [
+ "# Get parameters for model -- we can call this function to easily reset them\n",
+ "def get_parameters():\n",
+ " # And we'll create a network that approximately fits it\n",
+ " beta_0 = np.zeros((3,1)); # formerly theta_x0\n",
+ " omega_0 = np.zeros((3,1)); # formerly theta_x1\n",
+ " beta_1 = np.zeros((3,1)); # NOTE -- there are three outputs now (one for each class, so three output biases)\n",
+ " omega_1 = np.zeros((3,3)); # NOTE -- there are three outputs now (one for each class, so nine output weights, connecting 3 hidden units to 3 outputs)\n",
+ "\n",
+ " beta_0[0,0] = 0.3; beta_0[1,0] = -1.0; beta_0[2,0] = -0.5\n",
+ " omega_0[0,0] = -1.0; omega_0[1,0] = 1.8; omega_0[2,0] = 0.65\n",
+ " beta_1[0,0] = 2.0; beta_1[1,0] = -2; beta_1[2,0] = 0.0\n",
+ " omega_1[0,0] = -24.0; omega_1[0,1] = -8.0; omega_1[0,2] = 50.0\n",
+ " omega_1[1,0] = -2.0; omega_1[1,1] = 8.0; omega_1[1,2] = -30.0\n",
+ " omega_1[2,0] = 16.0; omega_1[2,1] = -8.0; omega_1[2,2] =-8\n",
+ "\n",
+ " return beta_0, omega_0, beta_1, omega_1"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "NRR67ri_1TzN"
+ },
+ "outputs": [],
+ "source": [
+ "# Utility function for plotting data\n",
+ "def plot_multiclass_classification(x_model, out_model, lambda_model, x_data = None, y_data = None, title= None):\n",
+ " # Make sure model data are 1D arrays\n",
+ " n_data = len(x_model)\n",
+ " n_class = 3\n",
+ " x_model = np.squeeze(x_model)\n",
+ " out_model = np.reshape(out_model, (n_class,n_data))\n",
+ " lambda_model = np.reshape(lambda_model, (n_class,n_data))\n",
+ "\n",
+ " fig, ax = plt.subplots(1,2)\n",
+ " fig.set_size_inches(7.0, 3.5)\n",
+ " fig.tight_layout(pad=3.0)\n",
+ " ax[0].plot(x_model,out_model[0,:],'r-')\n",
+ " ax[0].plot(x_model,out_model[1,:],'g-')\n",
+ " ax[0].plot(x_model,out_model[2,:],'b-')\n",
+ " ax[0].set_xlabel('Input, $x$'); ax[0].set_ylabel('Model outputs')\n",
+ " ax[0].set_xlim([0,1]);ax[0].set_ylim([-4,4])\n",
+ " if title is not None:\n",
+ " ax[0].set_title(title)\n",
+ " ax[1].plot(x_model,lambda_model[0,:],'r-')\n",
+ " ax[1].plot(x_model,lambda_model[1,:],'g-')\n",
+ " ax[1].plot(x_model,lambda_model[2,:],'b-')\n",
+ " ax[1].set_xlabel('Input, $x$'); ax[1].set_ylabel('$\\lambda$ or Pr(y=k|x)')\n",
+ " ax[1].set_xlim([0,1]);ax[1].set_ylim([-0.1,1.05])\n",
+ " if title is not None:\n",
+ " ax[1].set_title(title)\n",
+ " if x_data is not None:\n",
+ " for i in range(len(x_data)):\n",
+ " if y_data[i] ==0:\n",
+ " ax[1].plot(x_data[i],-0.05, 'r.')\n",
+ " if y_data[i] ==1:\n",
+ " ax[1].plot(x_data[i],-0.05, 'g.')\n",
+ " if y_data[i] ==2:\n",
+ " ax[1].plot(x_data[i],-0.05, 'b.')\n",
+ " plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "PsgLZwsPxauP"
+ },
+ "source": [
+ "# Multiclass classification\n",
+ "\n",
+ "For multiclass classification, the network must predict the probability of $K$ classes, using $K$ outputs. However, these probability must be non-negative and sum to one, and the network outputs can take arbitrary values. Hence, we pass the outputs through a softmax function which maps $K$ arbitrary values to $K$ non-negative values that sum to one."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "uFb8h-9IXnIe"
+ },
+ "outputs": [],
+ "source": [
+ "# Softmax function that maps a vector of arbitrary values to a vector of values that are positive and sum to one.\n",
+ "def softmax(model_out):\n",
+ " # This operation has to be done separately for every column of the input\n",
+ " # Compute exponentials of all the elements\n",
+ " # TODO: compute the softmax function (eq 5.22)\n",
+ " # Replace this skeleton code\n",
+ "\n",
+ " # Compute the exponential of the model outputs\n",
+ " exp_model_out = np.zeros_like(model_out) ;\n",
+ " # Compute the sum of the exponentials (denominator of equation 5.22)\n",
+ " sum_exp_model_out = np.zeros_like(model_out) ;\n",
+ " # Normalize the exponentials (np.matlib.repmat might be useful here)\n",
+ " softmax_model_out = np.ones_like(model_out)/ exp_model_out.shape[0]\n",
+ "\n",
+ " return softmax_model_out"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "VWzNOt1swFVd"
+ },
+ "outputs": [],
+ "source": [
+ "\n",
+ "# Let's create some 1D training data\n",
+ "x_train = np.array([0.09291784,0.46809093,0.93089486,0.67612654,0.73441752,0.86847339,\\\n",
+ " 0.49873225,0.51083168,0.18343972,0.99380898,0.27840809,0.38028817,\\\n",
+ " 0.12055708,0.56715537,0.92005746,0.77072270,0.85278176,0.05315950,\\\n",
+ " 0.87168699,0.58858043])\n",
+ "y_train = np.array([2,0,1,2,1,0,\\\n",
+ " 0,2,2,0,2,0,\\\n",
+ " 2,0,1,2,1,2, \\\n",
+ " 1,0])\n",
+ "\n",
+ "# Get parameters for the model\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "\n",
+ "# Define a range of input values\n",
+ "x_model = np.arange(0,1,0.01)\n",
+ "# Run the model to get values to plot and plot it.\n",
+ "model_out= shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
+ "lambda_model = softmax(model_out)\n",
+ "plot_multiclass_classification(x_model, model_out, lambda_model, x_train, y_train)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "MvVX6tl9AEXF"
+ },
+ "source": [
+ "The left is model output and the right is the model output after the softmax has been applied, so it now lies in the range [0,1] and represents the probability, that y=0 (red), 1 (green) and 2 (blue). The dots at the bottom show the training data with the same color scheme. So we want the red curve to be high where there are red dots, the green curve to be high where there are green dots, and the blue curve to be high where there are blue dots We'll compute the the likelihood and the negative log likelihood."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "YaLdRlEX0FkU"
+ },
+ "outputs": [],
+ "source": [
+ "# Return probability under categorical distribution for observed class y\n",
+ "# Just take value from row k of lambda param where y =k,\n",
+ "def categorical_distribution(y, lambda_param):\n",
+ " return np.array([lambda_param[row, i] for i, row in enumerate (y)])"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4TSL14dqHHbV"
+ },
+ "outputs": [],
+ "source": [
+ "# Let's double check we get the right answer before proceeding\n",
+ "print(\"Correct answer = %3.3f, Your answer = %3.3f\"%(0.2,categorical_distribution(np.array([[0]]),np.array([[0.2],[0.5],[0.3]]))))\n",
+ "print(\"Correct answer = %3.3f, Your answer = %3.3f\"%(0.5,categorical_distribution(np.array([[1]]),np.array([[0.2],[0.5],[0.3]]))))\n",
+ "print(\"Correct answer = %3.3f, Your answer = %3.3f\"%(0.3,categorical_distribution(np.array([[2]]),np.array([[0.2],[0.5],[0.3]]))))\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "R5z_0dzQMF35"
+ },
+ "source": [
+ "Now let's compute the likelihood using this function"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "zpS7o6liCx7f"
+ },
+ "outputs": [],
+ "source": [
+ "# Return the likelihood of all of the data under the model\n",
+ "def compute_likelihood(y_train, lambda_param):\n",
+ " # TODO -- compute the likelihood of the data -- the product of the categorical probabilities for each data point\n",
+ " # Top line of equation 5.3 in the notes\n",
+ " # You will need np.prod() and the categorical_distribution function you used above\n",
+ " # Replace the line below\n",
+ " likelihood = 0\n",
+ "\n",
+ " return likelihood"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "1hQxBLoVNlr2"
+ },
+ "outputs": [],
+ "source": [
+ "# Let's test this\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "# Use our neural network to predict the parameters of the categorical distribution\n",
+ "model_out = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+ "lambda_train = softmax(model_out)\n",
+ "# Compute the likelihood\n",
+ "likelihood = compute_likelihood(y_train, lambda_train)\n",
+ "# Let's double check we get the right answer before proceeding\n",
+ "print(\"Correct answer = %9.9f, Your answer = %9.9f\"%(0.000000041,likelihood))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "HzphKgPfOvlk"
+ },
+ "source": [
+ "You can see that this gives a very small answer, even for this small 1D dataset, and with the model fitting quite well. This is because it is the product of several probabilities, which are all quite small themselves.\n",
+ "This will get out of hand pretty quickly with real datasets -- the likelihood will get so small that we can't represent it with normal finite-precision math\n",
+ "\n",
+ "This is why we use negative log likelihood"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "dsT0CWiKBmTV"
+ },
+ "outputs": [],
+ "source": [
+ "# Return the negative log likelihood of the data under the model\n",
+ "def compute_negative_log_likelihood(y_train, lambda_param):\n",
+ " # TODO -- compute the negative log likelihood of the data -- don't use the likelihood function above -- compute the negative sum of the log probabilities\n",
+ " # You will need np.sum(), np.log()\n",
+ " # Replace the line below\n",
+ " nll = 0\n",
+ "\n",
+ " return nll"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "nVxUXg9rQmwI"
+ },
+ "outputs": [],
+ "source": [
+ "# Let's test this\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "# Use our neural network to predict the parameters of the categorical distribution\n",
+ "model_out = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+ "# Pass the outputs through the softmax function\n",
+ "lambda_train = softmax(model_out)\n",
+ "# Compute the negative log likelihood\n",
+ "nll = compute_negative_log_likelihood(y_train, lambda_train)\n",
+ "# Let's double check we get the right answer before proceeding\n",
+ "print(\"Correct answer = %9.9f, Your answer = %9.9f\"%(17.015457867,nll))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "OgcRojvPWh4V"
+ },
+ "source": [
+ "Now let's investigate finding the maximum likelihood / minimum negative log likelihood solution. For simplicity, we'll assume that all the parameters are fixed except one and look at how the likelihood and negative log likelihood change as we manipulate the last parameter. We'll start with overall y_offset, $\\beta_1$ (formerly $\\phi_0$)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "pFKtDaAeVU4U"
+ },
+ "outputs": [],
+ "source": [
+ "# Define a range of values for the parameter\n",
+ "beta_1_vals = np.arange(-2,6.0,0.1)\n",
+ "# Create some arrays to store the likelihoods, negative log likelihoods\n",
+ "likelihoods = np.zeros_like(beta_1_vals)\n",
+ "nlls = np.zeros_like(beta_1_vals)\n",
+ "\n",
+ "# Initialise the parameters\n",
+ "beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
+ "for count in range(len(beta_1_vals)):\n",
+ " # Set the value for the parameter\n",
+ " beta_1[0,0] = beta_1_vals[count]\n",
+ " # Run the network with new parameters\n",
+ " model_out = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
+ " lambda_train = softmax(model_out)\n",
+ " # Compute and store the two values\n",
+ " likelihoods[count] = compute_likelihood(y_train,lambda_train)\n",
+ " nlls[count] = compute_negative_log_likelihood(y_train, lambda_train)\n",
+ " # Draw the model for every 20th parameter setting\n",
+ " if count % 20 == 0:\n",
+ " # Run the model to get values to plot and plot it.\n",
+ " model_out = shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
+ " lambda_model = softmax(model_out)\n",
+ " plot_multiclass_classification(x_model, model_out, lambda_model, x_train, y_train, title=\"beta1[0,0]=%3.3f\"%(beta_1[0,0]))\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "UHXeTa9MagO6"
+ },
+ "outputs": [],
+ "source": [
+ "# Now let's plot the likelihood and negative log likelihood as a function of the value of the offset beta1\n",
+ "fig, ax = plt.subplots()\n",
+ "fig.tight_layout(pad=5.0)\n",
+ "likelihood_color = 'tab:red'\n",
+ "nll_color = 'tab:blue'\n",
+ "\n",
+ "\n",
+ "ax.set_xlabel('beta_1[0, 0]')\n",
+ "ax.set_ylabel('likelihood', color = likelihood_color)\n",
+ "ax.plot(beta_1_vals, likelihoods, color = likelihood_color)\n",
+ "ax.tick_params(axis='y', labelcolor=likelihood_color)\n",
+ "\n",
+ "ax1 = ax.twinx()\n",
+ "ax1.plot(beta_1_vals, nlls, color = nll_color)\n",
+ "ax1.set_ylabel('negative log likelihood', color = nll_color)\n",
+ "ax1.tick_params(axis='y', labelcolor = nll_color)\n",
+ "\n",
+ "plt.axvline(x = beta_1_vals[np.argmax(likelihoods)], linestyle='dotted')\n",
+ "\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "aDEPhddNdN4u"
+ },
+ "outputs": [],
+ "source": [
+ "# Hopefully, you can see that the maximum of the likelihood fn is at the same position as the minimum negative log likelihood solution\n",
+ "# Let's check that:\n",
+ "print(\"Maximum likelihood = %f, at beta_1=%3.3f\"%( (likelihoods[np.argmax(likelihoods)],beta_1_vals[np.argmax(likelihoods)])))\n",
+ "print(\"Minimum negative log likelihood = %f, at beta_1=%3.3f\"%( (nlls[np.argmin(nlls)],beta_1_vals[np.argmin(nlls)])))\n",
+ "\n",
+ "# Plot the best model\n",
+ "beta_1[0,0] = beta_1_vals[np.argmin(nlls)]\n",
+ "model_out = shallow_nn(x_model, beta_0, omega_0, beta_1, omega_1)\n",
+ "lambda_model = softmax(model_out)\n",
+ "plot_multiclass_classification(x_model, model_out, lambda_model, x_train, y_train, title=\"beta1[0,0]=%3.3f\"%(beta_1[0,0]))\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "771G8N1Vk5A2"
+ },
+ "source": [
+ "They both give the same answer. But you can see from the likelihood above that the likelihood is very small unless the parameters are almost correct. So in practice, we would work with the negative log likelihood."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 6.1: Line search**\n",
+ "\n",
+ "This notebook investigates how to find the minimum of a 1D function using line search as described in Figure 6.10.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
+ ],
+ "metadata": {
+ "id": "el8l05WQEO46"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "xhmIOLiZELV_"
+ },
+ "outputs": [],
+ "source": [
+ "# import libraries\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's create a simple 1D function\n",
+ "def loss_function(phi):\n",
+ " return 1- 0.5 * np.exp(-(phi-0.65)*(phi-0.65)/0.1) - 0.45 *np.exp(-(phi-0.35)*(phi-0.35)/0.02)\n",
+ "\n",
+ "def draw_function(loss_function,a=None, b=None, c=None, d=None):\n",
+ " # Plot the function\n",
+ " phi_plot = np.arange(0,1,0.01);\n",
+ " fig,ax = plt.subplots()\n",
+ " ax.plot(phi_plot,loss_function(phi_plot),'r-')\n",
+ " ax.set_xlim(0,1); ax.set_ylim(0,1)\n",
+ " ax.set_xlabel('$\\phi$'); ax.set_ylabel('$L[\\phi]$')\n",
+ " if a is not None and b is not None and c is not None and d is not None:\n",
+ " plt.axvspan(a, d, facecolor='k', alpha=0.2)\n",
+ " ax.plot([a,a],[0,1],'b-')\n",
+ " ax.plot([b,b],[0,1],'b-')\n",
+ " ax.plot([c,c],[0,1],'b-')\n",
+ " ax.plot([d,d],[0,1],'b-')\n",
+ " plt.show()\n"
+ ],
+ "metadata": {
+ "id": "qFRe9POHF2le"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Draw this function\n",
+ "draw_function(loss_function)"
+ ],
+ "metadata": {
+ "id": "TXx1Tpd1Tl-I"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now lets create a line search procedure to find the minimum in the range 0,1"
+ ],
+ "metadata": {
+ "id": "QU5mdGvpTtEG"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def line_search(loss_function, thresh=.0001, max_iter = 10, draw_flag = False):\n",
+ "\n",
+ " # Initialize four points along the range we are going to search\n",
+ " a = 0\n",
+ " b = 0.33\n",
+ " c = 0.66\n",
+ " d = 1.0\n",
+ " n_iter = 0\n",
+ "\n",
+ " # While we haven't found the minimum closely enough\n",
+ " while np.abs(b-c) > thresh and n_iter < max_iter:\n",
+ " # Increment iteration counter (just to prevent an infinite loop)\n",
+ " n_iter = n_iter+1\n",
+ "\n",
+ " # Calculate all four points\n",
+ " lossa = loss_function(a)\n",
+ " lossb = loss_function(b)\n",
+ " lossc = loss_function(c)\n",
+ " lossd = loss_function(d)\n",
+ "\n",
+ " if draw_flag:\n",
+ " draw_function(loss_function, a,b,c,d)\n",
+ "\n",
+ " print('Iter %d, a=%3.3f, b=%3.3f, c=%3.3f, d=%3.3f'%(n_iter, a,b,c,d))\n",
+ "\n",
+ " # Rule #1 If the HEIGHT at point A is less than the HEIGHT at points B, C, and D then halve values of B, C, and D\n",
+ " # i.e. bring them closer to the original point\n",
+ " # TODO REPLACE THE BLOCK OF CODE BELOW WITH THIS RULE\n",
+ " if (0):\n",
+ " continue;\n",
+ "\n",
+ "\n",
+ " # Rule #2 If the HEIGHT at point b is less than the HEIGHT at point c then\n",
+ " # point d becomes point c, and\n",
+ " # point b becomes 1/3 between a and new d\n",
+ " # point c becomes 2/3 between a and new d\n",
+ " # TODO REPLACE THE BLOCK OF CODE BELOW WITH THIS RULE\n",
+ " if (0):\n",
+ " continue;\n",
+ "\n",
+ " # Rule #3 If the HEIGHT at point c is less than the HEIGHT at point b then\n",
+ " # point a becomes point b, and\n",
+ " # point b becomes 1/3 between new a and d\n",
+ " # point c becomes 2/3 between new a and d\n",
+ " # TODO REPLACE THE BLOCK OF CODE BELOW WITH THIS RULE\n",
+ " if(0):\n",
+ " continue\n",
+ "\n",
+ " # TODO -- FINAL SOLUTION IS AVERAGE OF B and C\n",
+ " # REPLACE THIS LINE\n",
+ " soln = 1\n",
+ "\n",
+ "\n",
+ " return soln"
+ ],
+ "metadata": {
+ "id": "K-NTHpAAHlCl"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "soln = line_search(loss_function, draw_flag=True)\n",
+ "print('Soln = %3.3f, loss = %3.3f'%(soln,loss_function(soln)))"
+ ],
+ "metadata": {
+ "id": "YVq6rmaWRD2M"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "tOLd0gtdRLLS"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
diff --git a/Notebooks/Chap06/6_2_Gradient_Descent.ipynb b/Notebooks/Chap06/6_2_Gradient_Descent.ipynb
new file mode 100644
index 0000000..21a96bb
--- /dev/null
+++ b/Notebooks/Chap06/6_2_Gradient_Descent.ipynb
@@ -0,0 +1,428 @@
+{
+ "cells": [
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "colab_type": "text",
+ "id": "view-in-github"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "el8l05WQEO46"
+ },
+ "source": [
+ "# **Notebook 6.2 Gradient descent**\n",
+ "\n",
+ "This notebook recreates the gradient descent algorithm as shown in figure 6.1.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "xhmIOLiZELV_"
+ },
+ "outputs": [],
+ "source": [
+ "# import libraries\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from matplotlib import cm\n",
+ "from matplotlib.colors import ListedColormap"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4cRkrh9MZ58Z"
+ },
+ "outputs": [],
+ "source": [
+ "# Let's create our training data 12 pairs {x_i, y_i}\n",
+ "# We'll try to fit the straight line model to these data\n",
+ "data = np.array([[0.03,0.19,0.34,0.46,0.78,0.81,1.08,1.18,1.39,1.60,1.65,1.90],\n",
+ " [0.67,0.85,1.05,1.00,1.40,1.50,1.30,1.54,1.55,1.68,1.73,1.60]])"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "WQUERmb2erAe"
+ },
+ "outputs": [],
+ "source": [
+ "# Let's define our model -- just a straight line with intercept phi[0] and slope phi[1]\n",
+ "def model(phi,x):\n",
+ " y_pred = phi[0]+phi[1] * x\n",
+ " return y_pred"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "qFRe9POHF2le"
+ },
+ "outputs": [],
+ "source": [
+ "# Draw model\n",
+ "def draw_model(data,model,phi,title=None):\n",
+ " x_model = np.arange(0,2,0.01)\n",
+ " y_model = model(phi,x_model)\n",
+ "\n",
+ " fix, ax = plt.subplots()\n",
+ " ax.plot(data[0,:],data[1,:],'bo')\n",
+ " ax.plot(x_model,y_model,'m-')\n",
+ " ax.set_xlim([0,2]);ax.set_ylim([0,2])\n",
+ " ax.set_xlabel('x'); ax.set_ylabel('y')\n",
+ " ax.set_aspect('equal')\n",
+ " if title is not None:\n",
+ " ax.set_title(title)\n",
+ " plt.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "TXx1Tpd1Tl-I"
+ },
+ "outputs": [],
+ "source": [
+ "# Initialize the parameters to some arbitrary values and draw the model\n",
+ "phi = np.zeros((2,1))\n",
+ "phi[0] = 0.6 # Intercept\n",
+ "phi[1] = -0.2 # Slope\n",
+ "draw_model(data,model,phi, \"Initial parameters\")\n"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "QU5mdGvpTtEG"
+ },
+ "source": [
+ "Now let's compute the sum of squares loss for the training data"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "I7dqTY2Gg7CR"
+ },
+ "outputs": [],
+ "source": [
+ "def compute_loss(data_x, data_y, model, phi):\n",
+ " # TODO -- Write this function -- replace the line below\n",
+ " # First make model predictions from data x\n",
+ " # Then compute the squared difference between the predictions and true y values\n",
+ " # Then sum them all and return\n",
+ " pred_y = 0\n",
+ " loss = 0\n",
+ "\n",
+ " return loss"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "eB5DQvU5hYNx"
+ },
+ "source": [
+ "Let's just test that we got that right"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Ty05UtEEg9tc"
+ },
+ "outputs": [],
+ "source": [
+ "loss = compute_loss(data[0,:],data[1,:],model,np.array([[0.6],[-0.2]]))\n",
+ "print('Your loss = %3.3f, Correct loss = %3.3f'%(loss, 12.367))"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "F3trnavPiHpH"
+ },
+ "source": [
+ "Now let's plot the whole loss function"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "K-NTHpAAHlCl"
+ },
+ "outputs": [],
+ "source": [
+ "def draw_loss_function(compute_loss, data, model, phi_iters = None):\n",
+ " # Define pretty colormap\n",
+ " my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ " my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ " r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ " g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ " b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ " my_colormap = ListedColormap(np.vstack((r,g,b)).transpose()/255.0)\n",
+ "\n",
+ " # Make grid of intercept/slope values to plot\n",
+ " intercepts_mesh, slopes_mesh = np.meshgrid(np.arange(0.0,2.0,0.02), np.arange(-1.0,1.0,0.002))\n",
+ " loss_mesh = np.zeros_like(slopes_mesh)\n",
+ " # Compute loss for every set of parameters\n",
+ " for idslope, slope in np.ndenumerate(slopes_mesh):\n",
+ " loss_mesh[idslope] = compute_loss(data[0,:], data[1,:], model, np.array([[intercepts_mesh[idslope]], [slope]]))\n",
+ "\n",
+ " fig,ax = plt.subplots()\n",
+ " fig.set_size_inches(8,8)\n",
+ " ax.contourf(intercepts_mesh,slopes_mesh,loss_mesh,256,cmap=my_colormap)\n",
+ " ax.contour(intercepts_mesh,slopes_mesh,loss_mesh,40,colors=['#80808080'])\n",
+ " if phi_iters is not None:\n",
+ " ax.plot(phi_iters[0,:], phi_iters[1,:],'go-')\n",
+ " ax.set_ylim([1,-1])\n",
+ " ax.set_xlabel('Intercept $\\phi_{0}$'); ax.set_ylabel('Slope, $\\phi_{1}$')\n",
+ " plt.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "l8HbvIupnTME"
+ },
+ "outputs": [],
+ "source": [
+ "draw_loss_function(compute_loss, data, model)"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "s9Duf05WqqSC"
+ },
+ "source": [
+ "Now let's compute the gradient vector for a given set of parameters:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\frac{\\partial L}{\\partial \\boldsymbol\\phi} = \\begin{bmatrix}\\frac{\\partial L}{\\partial \\phi_0} \\\\\\frac{\\partial L}{\\partial \\phi_1} \\end{bmatrix}.\n",
+ "\\end{equation}"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "UpswmkL2qwBT"
+ },
+ "outputs": [],
+ "source": [
+ "# These are in the lecture slides and notes, but worth trying to calculate them yourself to\n",
+ "# check that you get them right. Write out the expression for the sum of squares loss and take the\n",
+ "# derivative with respect to phi0 and phi1\n",
+ "def compute_gradient(data_x, data_y, phi):\n",
+ " # TODO -- write this function, replacing the lines below\n",
+ " dl_dphi0 = 0.0\n",
+ " dl_dphi1 = 0.0\n",
+ "\n",
+ " # Return the gradient\n",
+ " return np.array([[dl_dphi0],[dl_dphi1]])"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "RS1nEcYVuEAM"
+ },
+ "source": [
+ "We can check we got this right using a trick known as **finite differences**. If we evaluate the function and then change one of the parameters by a very small amount and normalize by that amount, we get an approximation to the gradient, so:\n",
+ "\n",
+ "\\begin{align}\n",
+ "\\frac{\\partial L}{\\partial \\phi_{0}}&\\approx & \\frac{L[\\phi_0+\\delta, \\phi_1]-L[\\phi_0, \\phi_1]}{\\delta}\\\\\n",
+ "\\frac{\\partial L}{\\partial \\phi_{1}}&\\approx & \\frac{L[\\phi_0, \\phi_1+\\delta]-L[\\phi_0, \\phi_1]}{\\delta}\n",
+ "\\end{align}\n",
+ "\n",
+ "We can't do this when there are many parameters; for a million parameters, we would have to evaluate the loss function one million plus one times, and usually computing the gradients directly is much more efficient."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "QuwAHN7yt-gi"
+ },
+ "outputs": [],
+ "source": [
+ "# Compute the gradient using your function\n",
+ "gradient = compute_gradient(data[0,:],data[1,:], phi)\n",
+ "print(\"Your gradients: (%3.3f,%3.3f)\"%(gradient[0],gradient[1]))\n",
+ "# Approximate the gradients with finite differences\n",
+ "delta = 0.0001\n",
+ "dl_dphi0_est = (compute_loss(data[0,:],data[1,:],model,phi+np.array([[delta],[0]])) - \\\n",
+ " compute_loss(data[0,:],data[1,:],model,phi))/delta\n",
+ "dl_dphi1_est = (compute_loss(data[0,:],data[1,:],model,phi+np.array([[0],[delta]])) - \\\n",
+ " compute_loss(data[0,:],data[1,:],model,phi))/delta\n",
+ "print(\"Approx gradients: (%3.3f,%3.3f)\"%(dl_dphi0_est,dl_dphi1_est))\n",
+ "# There might be small differences in the last significant figure because finite gradients is an approximation\n"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "5EIjMM9Fw2eT"
+ },
+ "source": [
+ "Now we are ready to perform gradient descent. We'll need to use our line search routine from notebook 6.1, which I've reproduced here plus the helper function loss_function_1D that maps the search along the negative gradient direction in 2D space to a 1D problem (distance along this direction)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "XrJ2gQjfw1XP"
+ },
+ "outputs": [],
+ "source": [
+ "def loss_function_1D(dist_prop, data, model, phi_start, search_direction):\n",
+ " # Return the loss after moving this far\n",
+ " return compute_loss(data[0,:], data[1,:], model, phi_start+ search_direction * dist_prop)\n",
+ "\n",
+ "def line_search(data, model, phi, gradient, thresh=.00001, max_dist = 0.1, max_iter = 15, verbose=False):\n",
+ " # Initialize four points along the range we are going to search\n",
+ " a = 0\n",
+ " b = 0.33 * max_dist\n",
+ " c = 0.66 * max_dist\n",
+ " d = 1.0 * max_dist\n",
+ " n_iter = 0\n",
+ "\n",
+ " # While we haven't found the minimum closely enough\n",
+ " while np.abs(b-c) > thresh and n_iter < max_iter:\n",
+ " # Increment iteration counter (just to prevent an infinite loop)\n",
+ " n_iter = n_iter+1\n",
+ " # Calculate all four points\n",
+ " lossa = loss_function_1D(a, data, model, phi,gradient)\n",
+ " lossb = loss_function_1D(b, data, model, phi,gradient)\n",
+ " lossc = loss_function_1D(c, data, model, phi,gradient)\n",
+ " lossd = loss_function_1D(d, data, model, phi,gradient)\n",
+ "\n",
+ " if verbose:\n",
+ " print('Iter %d, a=%3.3f, b=%3.3f, c=%3.3f, d=%3.3f'%(n_iter, a,b,c,d))\n",
+ " print('a %f, b%f, c%f, d%f'%(lossa,lossb,lossc,lossd))\n",
+ "\n",
+ " # Rule #1 If point A is less than points B, C, and D then halve points B,C, and D\n",
+ " if np.argmin((lossa,lossb,lossc,lossd))==0:\n",
+ " b = b/2\n",
+ " c = c/2\n",
+ " d = d/2\n",
+ " continue;\n",
+ "\n",
+ " # Rule #2 If point b is less than point c then\n",
+ " # point d becomes point c, and\n",
+ " # point b becomes 1/3 between a and new d\n",
+ " # point c becomes 2/3 between a and new d\n",
+ " if lossb < lossc:\n",
+ " d = c\n",
+ " b = a+ (d-a)/3\n",
+ " c = a+ 2*(d-a)/3\n",
+ " continue\n",
+ "\n",
+ " # Rule #2 If point c is less than point b then\n",
+ " # point a becomes point b, and\n",
+ " # point b becomes 1/3 between new a and d\n",
+ " # point c becomes 2/3 between new a and d\n",
+ " a = b\n",
+ " b = a+ (d-a)/3\n",
+ " c = a+ 2*(d-a)/3\n",
+ "\n",
+ " # Return average of two middle points\n",
+ " return (b+c)/2.0"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "YVq6rmaWRD2M"
+ },
+ "outputs": [],
+ "source": [
+ "def gradient_descent_step(phi, data, model):\n",
+ " # TODO -- update Phi with the gradient descent step (equation 6.3)\n",
+ " # 1. Compute the gradient (you wrote this function above)\n",
+ " # 2. Find the best step size alpha using line search function (above) -- use negative gradient as going downhill\n",
+ " # 3. Update the parameters phi based on the gradient and the step size alpha.\n",
+ "\n",
+ " return phi"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "tOLd0gtdRLLS"
+ },
+ "outputs": [],
+ "source": [
+ "# Initialize the parameters and draw the model\n",
+ "n_steps = 10\n",
+ "phi_all = np.zeros((2,n_steps+1))\n",
+ "phi_all[0,0] = 1.6\n",
+ "phi_all[1,0] = -0.5\n",
+ "\n",
+ "# Measure loss and draw initial model\n",
+ "loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,0:1])\n",
+ "draw_model(data,model,phi_all[:,0:1], \"Initial parameters, Loss = %f\"%(loss))\n",
+ "\n",
+ "# Repeatedly take gradient descent steps\n",
+ "for c_step in range (n_steps):\n",
+ " # Do gradient descent step\n",
+ " phi_all[:,c_step+1:c_step+2] = gradient_descent_step(phi_all[:,c_step:c_step+1],data, model)\n",
+ " # Measure loss and draw model\n",
+ " loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,c_step+1:c_step+2])\n",
+ " draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
+ "\n",
+ "# Draw the trajectory on the loss function\n",
+ "draw_loss_function(compute_loss, data, model,phi_all)\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "include_colab_link": true,
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3",
+ "name": "python3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/Notebooks/Chap06/6_3_Stochastic_Gradient_Descent.ipynb b/Notebooks/Chap06/6_3_Stochastic_Gradient_Descent.ipynb
new file mode 100644
index 0000000..51b5055
--- /dev/null
+++ b/Notebooks/Chap06/6_3_Stochastic_Gradient_Descent.ipynb
@@ -0,0 +1,594 @@
+{
+ "cells": [
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "colab_type": "text",
+ "id": "view-in-github"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "el8l05WQEO46"
+ },
+ "source": [
+ "# **Notebook 6.3: Stochastic gradient descent**\n",
+ "\n",
+ "This notebook investigates gradient descent and stochastic gradient descent and recreates figure 6.5 from the book\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
+ "\n",
+ "\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "xhmIOLiZELV_"
+ },
+ "outputs": [],
+ "source": [
+ "# import libraries\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from matplotlib import cm\n",
+ "from matplotlib.colors import ListedColormap"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4cRkrh9MZ58Z"
+ },
+ "outputs": [],
+ "source": [
+ "# Let's create our training data of 30 pairs {x_i, y_i}\n",
+ "# We'll try to fit the Gabor model to these data\n",
+ "data = np.array([[-1.920e+00,-1.422e+01,1.490e+00,-1.940e+00,-2.389e+00,-5.090e+00,\n",
+ " -8.861e+00,3.578e+00,-6.010e+00,-6.995e+00,3.634e+00,8.743e-01,\n",
+ " -1.096e+01,4.073e-01,-9.467e+00,8.560e+00,1.062e+01,-1.729e-01,\n",
+ " 1.040e+01,-1.261e+01,1.574e-01,-1.304e+01,-2.156e+00,-1.210e+01,\n",
+ " -1.119e+01,2.902e+00,-8.220e+00,-1.179e+01,-8.391e+00,-4.505e+00],\n",
+ " [-1.051e+00,-2.482e-02,8.896e-01,-4.943e-01,-9.371e-01,4.306e-01,\n",
+ " 9.577e-03,-7.944e-02 ,1.624e-01,-2.682e-01,-3.129e-01,8.303e-01,\n",
+ " -2.365e-02,5.098e-01,-2.777e-01,3.367e-01,1.927e-01,-2.222e-01,\n",
+ " 6.352e-02,6.888e-03,3.224e-02,1.091e-02,-5.706e-01,-5.258e-02,\n",
+ " -3.666e-02,1.709e-01,-4.805e-02,2.008e-01,-1.904e-01,5.952e-01]])"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "WQUERmb2erAe"
+ },
+ "outputs": [],
+ "source": [
+ "# Let's define our model\n",
+ "def model(phi,x):\n",
+ " sin_component = np.sin(phi[0] + 0.06 * phi[1] * x)\n",
+ " gauss_component = np.exp(-(phi[0] + 0.06 * phi[1] * x) * (phi[0] + 0.06 * phi[1] * x) / 32)\n",
+ " y_pred= sin_component * gauss_component\n",
+ " return y_pred"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "qFRe9POHF2le"
+ },
+ "outputs": [],
+ "source": [
+ "# Draw model\n",
+ "def draw_model(data,model,phi,title=None):\n",
+ " x_model = np.arange(-15,15,0.1)\n",
+ " y_model = model(phi,x_model)\n",
+ "\n",
+ " fix, ax = plt.subplots()\n",
+ " ax.plot(data[0,:],data[1,:],'bo')\n",
+ " ax.plot(x_model,y_model,'m-')\n",
+ " ax.set_xlim([-15,15]);ax.set_ylim([-1,1])\n",
+ " ax.set_xlabel('x'); ax.set_ylabel('y')\n",
+ " if title is not None:\n",
+ " ax.set_title(title)\n",
+ " plt.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "TXx1Tpd1Tl-I"
+ },
+ "outputs": [],
+ "source": [
+ "# Initialize the parameters and draw the model\n",
+ "phi = np.zeros((2,1))\n",
+ "phi[0] = -5 # Horizontal offset\n",
+ "phi[1] = 25 # Frequency\n",
+ "draw_model(data,model,phi, \"Initial parameters\")\n"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "QU5mdGvpTtEG"
+ },
+ "source": [
+ "Now let's compute the sum of squares loss for the training data"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "I7dqTY2Gg7CR"
+ },
+ "outputs": [],
+ "source": [
+ "def compute_loss(data_x, data_y, model, phi):\n",
+ " # TODO -- Write this function -- replace the line below\n",
+ " # TODO -- First make model predictions from data x\n",
+ " # TODO -- Then compute the squared difference between the predictions and true y values\n",
+ " # TODO -- Then sum them all and return\n",
+ " loss = 0\n",
+ "\n",
+ " return loss"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "eB5DQvU5hYNx"
+ },
+ "source": [
+ "Let's just test that we got that right"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Ty05UtEEg9tc"
+ },
+ "outputs": [],
+ "source": [
+ "loss = compute_loss(data[0,:],data[1,:],model,np.array([[0.6],[-0.2]]))\n",
+ "print('Your loss = %3.3f, Correct loss = %3.3f'%(loss, 16.419))"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "F3trnavPiHpH"
+ },
+ "source": [
+ "Now let's plot the whole loss function"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "K-NTHpAAHlCl"
+ },
+ "outputs": [],
+ "source": [
+ "def draw_loss_function(compute_loss, data, model, phi_iters = None):\n",
+ " # Define pretty colormap\n",
+ " my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ " my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ " r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ " g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ " b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ " my_colormap = ListedColormap(np.vstack((r,g,b)).transpose()/255.0)\n",
+ "\n",
+ " # Make grid of offset/frequency values to plot\n",
+ " offsets_mesh, freqs_mesh = np.meshgrid(np.arange(-10,10.0,0.1), np.arange(2.5,22.5,0.1))\n",
+ " loss_mesh = np.zeros_like(freqs_mesh)\n",
+ " # Compute loss for every set of parameters\n",
+ " for idslope, slope in np.ndenumerate(freqs_mesh):\n",
+ " loss_mesh[idslope] = compute_loss(data[0,:], data[1,:], model, np.array([[offsets_mesh[idslope]], [slope]]))\n",
+ "\n",
+ " fig,ax = plt.subplots()\n",
+ " fig.set_size_inches(8,8)\n",
+ " ax.contourf(offsets_mesh,freqs_mesh,loss_mesh,256,cmap=my_colormap)\n",
+ " ax.contour(offsets_mesh,freqs_mesh,loss_mesh,20,colors=['#80808080'])\n",
+ " if phi_iters is not None:\n",
+ " ax.plot(phi_iters[0,:], phi_iters[1,:],'go-')\n",
+ " ax.set_ylim([2.5,22.5])\n",
+ " ax.set_xlabel('Offset $\\phi_{0}$'); ax.set_ylabel('Frequency, $\\phi_{1}$')\n",
+ " plt.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "l8HbvIupnTME"
+ },
+ "outputs": [],
+ "source": [
+ "draw_loss_function(compute_loss, data, model)"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "s9Duf05WqqSC"
+ },
+ "source": [
+ "Now let's compute the gradient vector for a given set of parameters:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\frac{\\partial L}{\\partial \\boldsymbol\\phi} = \\begin{bmatrix}\\frac{\\partial L}{\\partial \\phi_0} \\\\\\frac{\\partial L}{\\partial \\phi_1} \\end{bmatrix}.\n",
+ "\\end{equation}"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "UpswmkL2qwBT"
+ },
+ "outputs": [],
+ "source": [
+ "# These came from writing out the expression for the sum of squares loss and taking the\n",
+ "# derivative with respect to phi0 and phi1. It was a lot of hassle to get it right!\n",
+ "def gabor_deriv_phi0(data_x,data_y,phi0, phi1):\n",
+ " x = 0.06 * phi1 * data_x + phi0\n",
+ " y = data_y\n",
+ " cos_component = np.cos(x)\n",
+ " sin_component = np.sin(x)\n",
+ " gauss_component = np.exp(-0.5 * x *x / 16)\n",
+ " deriv = cos_component * gauss_component - sin_component * gauss_component * x / 16\n",
+ " deriv = 2* deriv * (sin_component * gauss_component - y)\n",
+ " return np.sum(deriv)\n",
+ "\n",
+ "def gabor_deriv_phi1(data_x, data_y,phi0, phi1):\n",
+ " x = 0.06 * phi1 * data_x + phi0\n",
+ " y = data_y\n",
+ " cos_component = np.cos(x)\n",
+ " sin_component = np.sin(x)\n",
+ " gauss_component = np.exp(-0.5 * x *x / 16)\n",
+ " deriv = 0.06 * data_x * cos_component * gauss_component - 0.06 * data_x*sin_component * gauss_component * x / 16\n",
+ " deriv = 2*deriv * (sin_component * gauss_component - y)\n",
+ " return np.sum(deriv)\n",
+ "\n",
+ "def compute_gradient(data_x, data_y, phi):\n",
+ " dl_dphi0 = gabor_deriv_phi0(data_x, data_y, phi[0],phi[1])\n",
+ " dl_dphi1 = gabor_deriv_phi1(data_x, data_y, phi[0],phi[1])\n",
+ " # Return the gradient\n",
+ " return np.array([[dl_dphi0],[dl_dphi1]])"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "RS1nEcYVuEAM"
+ },
+ "source": [
+ "We can check we got this right using a trick known as **finite differences**. If we evaluate the function and then change one of the parameters by a very small amount and normalize by that amount, we get an approximation to the gradient, so:\n",
+ "\n",
+ "\\begin{align}\n",
+ "\\frac{\\partial L}{\\partial \\phi_{0}}&\\approx & \\frac{L[\\phi_0+\\delta, \\phi_1]-L[\\phi_0, \\phi_1]}{\\delta}\\\\\n",
+ "\\frac{\\partial L}{\\partial \\phi_{1}}&\\approx & \\frac{L[\\phi_0, \\phi_1+\\delta]-L[\\phi_0, \\phi_1]}{\\delta}\n",
+ "\\end{align}\n",
+ "\n",
+ "We can't do this when there are many parameters; for a million parameters, we would have to evaluate the loss function two million times, and usually computing the gradients directly is much more efficient."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "QuwAHN7yt-gi"
+ },
+ "outputs": [],
+ "source": [
+ "# Compute the gradient using your function\n",
+ "gradient = compute_gradient(data[0,:],data[1,:], phi)\n",
+ "print(\"Your gradients: (%3.3f,%3.3f)\"%(gradient[0],gradient[1]))\n",
+ "# Approximate the gradients with finite differences\n",
+ "delta = 0.0001\n",
+ "dl_dphi0_est = (compute_loss(data[0,:],data[1,:],model,phi+np.array([[delta],[0]])) - \\\n",
+ " compute_loss(data[0,:],data[1,:],model,phi))/delta\n",
+ "dl_dphi1_est = (compute_loss(data[0,:],data[1,:],model,phi+np.array([[0],[delta]])) - \\\n",
+ " compute_loss(data[0,:],data[1,:],model,phi))/delta\n",
+ "print(\"Approx gradients: (%3.3f,%3.3f)\"%(dl_dphi0_est,dl_dphi1_est))\n"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "5EIjMM9Fw2eT"
+ },
+ "source": [
+ "Now we are ready to perform gradient descent. We'll need to use our line search routine from Notebook 6.1, which I've reproduced here plus the helper function loss_function_1D that converts from a 2D problem to a 1D problem"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "XrJ2gQjfw1XP"
+ },
+ "outputs": [],
+ "source": [
+ "def loss_function_1D(dist_prop, data, model, phi_start, gradient):\n",
+ " # Return the loss after moving this far\n",
+ " return compute_loss(data[0,:], data[1,:], model, phi_start+ gradient * dist_prop)\n",
+ "\n",
+ "def line_search(data, model, phi, gradient, thresh=.00001, max_dist = 0.1, max_iter = 15, verbose=False):\n",
+ " # Initialize four points along the range we are going to search\n",
+ " a = 0\n",
+ " b = 0.33 * max_dist\n",
+ " c = 0.66 * max_dist\n",
+ " d = 1.0 * max_dist\n",
+ " n_iter = 0\n",
+ "\n",
+ " # While we haven't found the minimum closely enough\n",
+ " while np.abs(b-c) > thresh and n_iter < max_iter:\n",
+ " # Increment iteration counter (just to prevent an infinite loop)\n",
+ " n_iter = n_iter+1\n",
+ " # Calculate all four points\n",
+ " lossa = loss_function_1D(a, data, model, phi,gradient)\n",
+ " lossb = loss_function_1D(b, data, model, phi,gradient)\n",
+ " lossc = loss_function_1D(c, data, model, phi,gradient)\n",
+ " lossd = loss_function_1D(d, data, model, phi,gradient)\n",
+ "\n",
+ " if verbose:\n",
+ " print('Iter %d, a=%3.3f, b=%3.3f, c=%3.3f, d=%3.3f'%(n_iter, a,b,c,d))\n",
+ " print('a %f, b%f, c%f, d%f'%(lossa,lossb,lossc,lossd))\n",
+ "\n",
+ " # Rule #1 If point A is less than points B, C, and D then halve points B,C, and D\n",
+ " if np.argmin((lossa,lossb,lossc,lossd))==0:\n",
+ " b = b/2\n",
+ " c = c/2\n",
+ " d = d/2\n",
+ " continue;\n",
+ "\n",
+ " # Rule #2 If point b is less than point c then\n",
+ " # point d becomes point c, and\n",
+ " # point b becomes 1/3 between a and new d\n",
+ " # point c becomes 2/3 between a and new d\n",
+ " if lossb < lossc:\n",
+ " d = c\n",
+ " b = a+ (d-a)/3\n",
+ " c = a+ 2*(d-a)/3\n",
+ " continue\n",
+ "\n",
+ " # Rule #2 If point c is less than point b then\n",
+ " # point a becomes point b, and\n",
+ " # point b becomes 1/3 between new a and d\n",
+ " # point c becomes 2/3 between new a and d\n",
+ " a = b\n",
+ " b = a+ (d-a)/3\n",
+ " c = a+ 2*(d-a)/3\n",
+ "\n",
+ " # Return average of two middle points\n",
+ " return (b+c)/2.0"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "YVq6rmaWRD2M"
+ },
+ "outputs": [],
+ "source": [
+ "def gradient_descent_step(phi, data, model):\n",
+ " # Step 1: Compute the gradient\n",
+ " gradient = compute_gradient(data[0,:],data[1,:], phi)\n",
+ " # Step 2: Update the parameters -- note we want to search in the negative (downhill direction)\n",
+ " alpha = line_search(data, model, phi, gradient*-1, max_dist = 2.0)\n",
+ " phi = phi - alpha * gradient\n",
+ " return phi"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "tOLd0gtdRLLS"
+ },
+ "outputs": [],
+ "source": [
+ "# Initialize the parameters\n",
+ "n_steps = 21\n",
+ "phi_all = np.zeros((2,n_steps+1))\n",
+ "phi_all[0,0] = -1.5\n",
+ "phi_all[1,0] = 8.5\n",
+ "\n",
+ "# Measure loss and draw initial model\n",
+ "loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,0:1])\n",
+ "draw_model(data,model,phi_all[:,0:1], \"Initial parameters, Loss = %f\"%(loss))\n",
+ "\n",
+ "for c_step in range (n_steps):\n",
+ " # Do gradient descent step\n",
+ " phi_all[:,c_step+1:c_step+2] = gradient_descent_step(phi_all[:,c_step:c_step+1],data, model)\n",
+ " # Measure loss and draw model every 4th step\n",
+ " if c_step % 4 == 0:\n",
+ " loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,c_step+1:c_step+2])\n",
+ " draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
+ "\n",
+ "draw_loss_function(compute_loss, data, model,phi_all)\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Oi8ZlH0ptLqA"
+ },
+ "outputs": [],
+ "source": [
+ "# TODO Experiment with starting the optimization in the previous cell in different places\n",
+ "# and show that it heads to a local minimum if we don't start it in the right valley"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4l-ueLk-oAxV"
+ },
+ "outputs": [],
+ "source": [
+ "def gradient_descent_step_fixed_learning_rate(phi, data, alpha):\n",
+ " # TODO -- fill in this routine so that we take a fixed size step of size alpha without using line search\n",
+ "\n",
+ " return phi"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "oi9MX_GRpM41"
+ },
+ "outputs": [],
+ "source": [
+ "# Initialize the parameters\n",
+ "n_steps = 21\n",
+ "phi_all = np.zeros((2,n_steps+1))\n",
+ "phi_all[0,0] = -1.5\n",
+ "phi_all[1,0] = 8.5\n",
+ "\n",
+ "# Measure loss and draw initial model\n",
+ "loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,0:1])\n",
+ "draw_model(data,model,phi_all[:,0:1], \"Initial parameters, Loss = %f\"%(loss))\n",
+ "\n",
+ "for c_step in range (n_steps):\n",
+ " # Do gradient descent step\n",
+ " phi_all[:,c_step+1:c_step+2] = gradient_descent_step_fixed_learning_rate(phi_all[:,c_step:c_step+1],data, alpha =0.2)\n",
+ " # Measure loss and draw model every 4th step\n",
+ " if c_step % 4 == 0:\n",
+ " loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,c_step+1:c_step+2])\n",
+ " draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
+ "\n",
+ "draw_loss_function(compute_loss, data, model,phi_all)\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "In6sQ5YCpMqn"
+ },
+ "outputs": [],
+ "source": [
+ "# TODO Experiment with the learning rate, alpha.\n",
+ "# What happens if you set it too large?\n",
+ "# What happens if you set it too small?"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "VKTC9-1Gpm3N"
+ },
+ "outputs": [],
+ "source": [
+ "def stochastic_gradient_descent_step(phi, data, alpha, batch_size):\n",
+ " # TODO -- fill in this routine so that we take a fixed size step of size alpha but only using a subset (batch) of the data\n",
+ " # at each step\n",
+ " # You can use the function np.random.permutation to generate a random permutation of the n_data = data.shape[1] indices\n",
+ " # and then just choose the first n=batch_size of these indices. Then compute the gradient update\n",
+ " # from just the data with these indices. More properly, you should sample without replacement, but this will do for now.\n",
+ "\n",
+ "\n",
+ " return phi"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "469OP_UHskJ4"
+ },
+ "outputs": [],
+ "source": [
+ "# Set the random number generator so you always get same numbers (disable if you don't want this)\n",
+ "np.random.seed(1)\n",
+ "# Initialize the parameters\n",
+ "n_steps = 41\n",
+ "phi_all = np.zeros((2,n_steps+1))\n",
+ "phi_all[0,0] = 3.5\n",
+ "phi_all[1,0] = 6.5\n",
+ "\n",
+ "# Measure loss and draw initial model\n",
+ "loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,0:1])\n",
+ "draw_model(data,model,phi_all[:,0:1], \"Initial parameters, Loss = %f\"%(loss))\n",
+ "\n",
+ "for c_step in range (n_steps):\n",
+ " # Do gradient descent step\n",
+ " phi_all[:,c_step+1:c_step+2] = stochastic_gradient_descent_step(phi_all[:,c_step:c_step+1],data, alpha =0.8, batch_size=5)\n",
+ " # Measure loss and draw model every 8th step\n",
+ " if c_step % 8 == 0:\n",
+ " loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,c_step+1:c_step+2])\n",
+ " draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
+ "\n",
+ "draw_loss_function(compute_loss, data, model,phi_all)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "LxE2kTa3s29p"
+ },
+ "outputs": [],
+ "source": [
+ "# TODO -- Experiment with different learning rates, starting points, batch sizes, number of steps. Get a feel for this."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "lw4QPOaQTh5e"
+ },
+ "outputs": [],
+ "source": [
+ "# TODO -- Add a learning rate schedule. Reduce the learning rate by a factor of beta every M iterations"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "authorship_tag": "ABX9TyNk5FN4qlw3pk8BwDVWw1jN",
+ "include_colab_link": true,
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3",
+ "name": "python3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/Notebooks/Chap06/6_4_Momentum.ipynb b/Notebooks/Chap06/6_4_Momentum.ipynb
new file mode 100644
index 0000000..e7f7c9d
--- /dev/null
+++ b/Notebooks/Chap06/6_4_Momentum.ipynb
@@ -0,0 +1,389 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 6.4: Momentum**\n",
+ "\n",
+ "This notebook investigates the use of momentum as illustrated in figure 6.7 from the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
+ "\n",
+ "\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "el8l05WQEO46"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "xhmIOLiZELV_"
+ },
+ "outputs": [],
+ "source": [
+ "# import libraries\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from matplotlib import cm\n",
+ "from matplotlib.colors import ListedColormap"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's create our training data of 30 pairs {x_i, y_i}\n",
+ "# We'll try to fit the Gabor model to these data\n",
+ "data = np.array([[-1.920e+00,-1.422e+01,1.490e+00,-1.940e+00,-2.389e+00,-5.090e+00,\n",
+ " -8.861e+00,3.578e+00,-6.010e+00,-6.995e+00,3.634e+00,8.743e-01,\n",
+ " -1.096e+01,4.073e-01,-9.467e+00,8.560e+00,1.062e+01,-1.729e-01,\n",
+ " 1.040e+01,-1.261e+01,1.574e-01,-1.304e+01,-2.156e+00,-1.210e+01,\n",
+ " -1.119e+01,2.902e+00,-8.220e+00,-1.179e+01,-8.391e+00,-4.505e+00],\n",
+ " [-1.051e+00,-2.482e-02,8.896e-01,-4.943e-01,-9.371e-01,4.306e-01,\n",
+ " 9.577e-03,-7.944e-02 ,1.624e-01,-2.682e-01,-3.129e-01,8.303e-01,\n",
+ " -2.365e-02,5.098e-01,-2.777e-01,3.367e-01,1.927e-01,-2.222e-01,\n",
+ " 6.352e-02,6.888e-03,3.224e-02,1.091e-02,-5.706e-01,-5.258e-02,\n",
+ " -3.666e-02,1.709e-01,-4.805e-02,2.008e-01,-1.904e-01,5.952e-01]])"
+ ],
+ "metadata": {
+ "id": "4cRkrh9MZ58Z"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's define our model\n",
+ "def model(phi,x):\n",
+ " sin_component = np.sin(phi[0] + 0.06 * phi[1] * x)\n",
+ " gauss_component = np.exp(-(phi[0] + 0.06 * phi[1] * x) * (phi[0] + 0.06 * phi[1] * x) / 32)\n",
+ " y_pred= sin_component * gauss_component\n",
+ " return y_pred"
+ ],
+ "metadata": {
+ "id": "WQUERmb2erAe"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Draw model\n",
+ "def draw_model(data,model,phi,title=None):\n",
+ " x_model = np.arange(-15,15,0.1)\n",
+ " y_model = model(phi,x_model)\n",
+ "\n",
+ " fix, ax = plt.subplots()\n",
+ " ax.plot(data[0,:],data[1,:],'bo')\n",
+ " ax.plot(x_model,y_model,'m-')\n",
+ " ax.set_xlim([-15,15]);ax.set_ylim([-1,1])\n",
+ " ax.set_xlabel('x'); ax.set_ylabel('y')\n",
+ " if title is not None:\n",
+ " ax.set_title(title)\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "qFRe9POHF2le"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Initialize the parameters and draw the model\n",
+ "phi = np.zeros((2,1))\n",
+ "phi[0] = -5 # Horizontal offset\n",
+ "phi[1] = 25 # Frequency\n",
+ "draw_model(data,model,phi, \"Initial parameters\")\n"
+ ],
+ "metadata": {
+ "id": "TXx1Tpd1Tl-I"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's compute the sum of squares loss for the training data and plot the loss function"
+ ],
+ "metadata": {
+ "id": "QU5mdGvpTtEG"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def compute_loss(data_x, data_y, model, phi):\n",
+ " pred_y = model(phi, data_x)\n",
+ " loss = np.sum((pred_y-data_y)*(pred_y-data_y))\n",
+ " return loss\n",
+ "\n",
+ "def draw_loss_function(compute_loss, data, model, phi_iters = None):\n",
+ " # Define pretty colormap\n",
+ " my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ " my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ " r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ " g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ " b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ " my_colormap = ListedColormap(np.vstack((r,g,b)).transpose()/255.0)\n",
+ "\n",
+ " # Make grid of offset/frequency values to plot\n",
+ " offsets_mesh, freqs_mesh = np.meshgrid(np.arange(-10,10.0,0.1), np.arange(2.5,22.5,0.1))\n",
+ " loss_mesh = np.zeros_like(freqs_mesh)\n",
+ " # Compute loss for every set of parameters\n",
+ " for idslope, slope in np.ndenumerate(freqs_mesh):\n",
+ " loss_mesh[idslope] = compute_loss(data[0,:], data[1,:], model, np.array([[offsets_mesh[idslope]], [slope]]))\n",
+ "\n",
+ " fig,ax = plt.subplots()\n",
+ " fig.set_size_inches(8,8)\n",
+ " ax.contourf(offsets_mesh,freqs_mesh,loss_mesh,256,cmap=my_colormap)\n",
+ " ax.contour(offsets_mesh,freqs_mesh,loss_mesh,20,colors=['#80808080'])\n",
+ " if phi_iters is not None:\n",
+ " ax.plot(phi_iters[0,:], phi_iters[1,:],'go-')\n",
+ " ax.set_ylim([2.5,22.5])\n",
+ " ax.set_xlabel('Offset $\\phi_{0}$'); ax.set_ylabel('Frequency, $\\phi_{1}$')\n",
+ " plt.show()\n",
+ "\n",
+ "draw_loss_function(compute_loss, data, model)"
+ ],
+ "metadata": {
+ "id": "I7dqTY2Gg7CR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "As before, we compute the gradient vector for a given set of parameters:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\frac{\\partial L}{\\partial \\boldsymbol\\phi} = \\begin{bmatrix}\\frac{\\partial L}{\\partial \\phi_0} \\\\\\frac{\\partial L}{\\partial \\phi_1} \\end{bmatrix}.\n",
+ "\\end{equation}"
+ ],
+ "metadata": {
+ "id": "s9Duf05WqqSC"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# These came from writing out the expression for the sum of squares loss and taking the\n",
+ "# derivative with respect to phi0 and phi1. It was a lot of hassle to get it right!\n",
+ "def gabor_deriv_phi0(data_x,data_y,phi0, phi1):\n",
+ " x = 0.06 * phi1 * data_x + phi0\n",
+ " y = data_y\n",
+ " cos_component = np.cos(x)\n",
+ " sin_component = np.sin(x)\n",
+ " gauss_component = np.exp(-0.5 * x *x / 16)\n",
+ " deriv = cos_component * gauss_component - sin_component * gauss_component * x / 16\n",
+ " deriv = 2* deriv * (sin_component * gauss_component - y)\n",
+ " return np.sum(deriv)\n",
+ "\n",
+ "def gabor_deriv_phi1(data_x, data_y,phi0, phi1):\n",
+ " x = 0.06 * phi1 * data_x + phi0\n",
+ " y = data_y\n",
+ " cos_component = np.cos(x)\n",
+ " sin_component = np.sin(x)\n",
+ " gauss_component = np.exp(-0.5 * x *x / 16)\n",
+ " deriv = 0.06 * data_x * cos_component * gauss_component - 0.06 * data_x*sin_component * gauss_component * x / 16\n",
+ " deriv = 2*deriv * (sin_component * gauss_component - y)\n",
+ " return np.sum(deriv)\n",
+ "\n",
+ "def compute_gradient(data_x, data_y, phi):\n",
+ " dl_dphi0 = gabor_deriv_phi0(data_x, data_y, phi[0],phi[1])\n",
+ " dl_dphi1 = gabor_deriv_phi1(data_x, data_y, phi[0],phi[1])\n",
+ " # Return the gradient\n",
+ " return np.array([[dl_dphi0],[dl_dphi1]])"
+ ],
+ "metadata": {
+ "id": "UpswmkL2qwBT"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's first run standard stochastic gradient descent."
+ ],
+ "metadata": {
+ "id": "7Tv3d4zqAdZR"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set the random number generator so you always get same numbers (disable if you don't want this)\n",
+ "np.random.seed(1)\n",
+ "# Initialize the parameters\n",
+ "n_steps = 81\n",
+ "batch_size = 5\n",
+ "alpha = 0.6\n",
+ "phi_all = np.zeros((2,n_steps+1))\n",
+ "phi_all[0,0] = -1.5\n",
+ "phi_all[1,0] = 6.5\n",
+ "\n",
+ "# Measure loss and draw initial model\n",
+ "loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,0:1])\n",
+ "draw_model(data,model,phi_all[:,0:1], \"Initial parameters, Loss = %f\"%(loss))\n",
+ "\n",
+ "for c_step in range (n_steps):\n",
+ " # Choose random batch indices\n",
+ " batch_index = np.random.permutation(data.shape[1])[0:batch_size]\n",
+ " # Compute the gradient\n",
+ " gradient = compute_gradient(data[0,batch_index], data[1,batch_index], phi_all[:,c_step:c_step+1] )\n",
+ " # Update the parameters\n",
+ " phi_all[:,c_step+1:c_step+2] = phi_all[:,c_step:c_step+1] - alpha * gradient\n",
+ "\n",
+ "loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,c_step+1:c_step+2])\n",
+ "draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
+ "draw_loss_function(compute_loss, data, model,phi_all)"
+ ],
+ "metadata": {
+ "id": "469OP_UHskJ4"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's add momentum (equation 6.11)"
+ ],
+ "metadata": {
+ "id": "nMILovgMFpdI"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set the random number generator so you always get same numbers (disable if you don't want this)\n",
+ "np.random.seed(1)\n",
+ "# Initialize the parameters\n",
+ "n_steps = 81\n",
+ "batch_size = 5\n",
+ "alpha = 0.6\n",
+ "beta = 0.6\n",
+ "momentum = np.zeros([2,1])\n",
+ "phi_all = np.zeros((2,n_steps+1))\n",
+ "phi_all[0,0] = -1.5\n",
+ "phi_all[1,0] = 6.5\n",
+ "\n",
+ "# Measure loss and draw initial model\n",
+ "loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,0:1])\n",
+ "draw_model(data,model,phi_all[:,0:1], \"Initial parameters, Loss = %f\"%(loss))\n",
+ "\n",
+ "for c_step in range (n_steps):\n",
+ " # Choose random batch indices\n",
+ " batch_index = np.random.permutation(data.shape[1])[0:batch_size]\n",
+ " # Compute the gradient\n",
+ " gradient = compute_gradient(data[0,batch_index], data[1,batch_index], phi_all[:,c_step:c_step+1])\n",
+ " # TODO -- calculate momentum - replace the line below\n",
+ " momentum = np.zeros([2,1])\n",
+ "\n",
+ " # Update the parameters\n",
+ " phi_all[:,c_step+1:c_step+2] = phi_all[:,c_step:c_step+1] - alpha * momentum\n",
+ "\n",
+ "loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,c_step+1:c_step+2])\n",
+ "draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
+ "draw_loss_function(compute_loss, data, model,phi_all)"
+ ],
+ "metadata": {
+ "id": "dWBU8ZbSFny9"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Finally, we'll try Nesterov momentum"
+ ],
+ "metadata": {
+ "id": "nYIAomA-KPkU"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set the random number generator so you always get same numbers (disable if you don't want this)\n",
+ "np.random.seed(1)\n",
+ "# Initialize the parameters\n",
+ "n_steps = 81\n",
+ "batch_size = 5\n",
+ "alpha = 0.6\n",
+ "beta = 0.6\n",
+ "momentum = np.zeros([2,1])\n",
+ "phi_all = np.zeros((2,n_steps+1))\n",
+ "phi_all[0,0] = -1.5\n",
+ "phi_all[1,0] = 6.5\n",
+ "\n",
+ "# Measure loss and draw initial model\n",
+ "loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,0:1])\n",
+ "draw_model(data,model,phi_all[:,0:1], \"Initial parameters, Loss = %f\"%(loss))\n",
+ "\n",
+ "for c_step in range (n_steps):\n",
+ " # Choose random batch indices\n",
+ " batch_index = np.random.permutation(data.shape[1])[0:batch_size]\n",
+ " # TODO -- calculate Nesterov momentum - replace the lines below\n",
+ " gradient = np.zeros([2,1])\n",
+ " momentum = np.zeros([2,1])\n",
+ "\n",
+ " # Update the parameters\n",
+ " phi_all[:,c_step+1:c_step+2] = phi_all[:,c_step:c_step+1] - alpha * momentum\n",
+ "\n",
+ "loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,c_step+1:c_step+2])\n",
+ "draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
+ "draw_loss_function(compute_loss, data, model,phi_all)"
+ ],
+ "metadata": {
+ "id": "XtwWeCZ5HLLh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Note that for this case, Nesterov momentum does not improve the result."
+ ],
+ "metadata": {
+ "id": "F-As4hS8s2nm"
+ }
+ }
+ ]
+}
diff --git a/Notebooks/Chap06/6_5_Adam.ipynb b/Notebooks/Chap06/6_5_Adam.ipynb
new file mode 100644
index 0000000..b13fb1d
--- /dev/null
+++ b/Notebooks/Chap06/6_5_Adam.ipynb
@@ -0,0 +1,287 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 6.5: Adam**\n",
+ "\n",
+ "This notebook investigates the Adam algorithm as illustrated in figure 6.9 from the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "ysg9OHZq07YC"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Hi_t5nCk01tx"
+ },
+ "outputs": [],
+ "source": [
+ "# import libraries\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from matplotlib.colors import ListedColormap"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define function that we wish to find the minimum of (normally would be defined implicitly by data and loss)\n",
+ "def loss(phi0, phi1):\n",
+ " height = np.exp(-0.5 * (phi1 * phi1)*4.0)\n",
+ " height = height * np. exp(-0.5* (phi0-0.7) *(phi0-0.7)/4.0)\n",
+ " return 1.0-height\n",
+ "\n",
+ "# Compute the gradients of this function (for simplicity, I just used finite differences)\n",
+ "def get_loss_gradient(phi0, phi1):\n",
+ " delta_phi = 0.00001;\n",
+ " gradient = np.zeros((2,1));\n",
+ " gradient[0] = (loss(phi0+delta_phi/2.0, phi1) - loss(phi0-delta_phi/2.0, phi1))/delta_phi\n",
+ " gradient[1] = (loss(phi0, phi1+delta_phi/2.0) - loss(phi0, phi1-delta_phi/2.0))/delta_phi\n",
+ " return gradient[:,0];\n",
+ "\n",
+ "# Compute the loss function at a range of values of phi0 and phi1 for plotting\n",
+ "def get_loss_function_for_plot():\n",
+ " grid_values = np.arange(-1.0,1.0,0.01);\n",
+ " phi0mesh, phi1mesh = np.meshgrid(grid_values, grid_values)\n",
+ " loss_function = np.zeros((grid_values.size, grid_values.size))\n",
+ " for idphi0, phi0 in enumerate(grid_values):\n",
+ " for idphi1, phi1 in enumerate(grid_values):\n",
+ " loss_function[idphi0, idphi1] = loss(phi1,phi0)\n",
+ " return loss_function, phi0mesh, phi1mesh"
+ ],
+ "metadata": {
+ "id": "GTrgOKhp16zw"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define fancy colormap\n",
+ "my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ "my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ "r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ "g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ "b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ "my_colormap_vals = np.vstack((r,g,b)).transpose()/255.0\n",
+ "my_colormap = ListedColormap(my_colormap_vals)\n",
+ "\n",
+ "# Plotting function\n",
+ "def draw_function(phi0mesh, phi1mesh, loss_function, my_colormap, opt_path):\n",
+ " fig = plt.figure();\n",
+ " ax = plt.axes();\n",
+ " fig.set_size_inches(7,7)\n",
+ " ax.contourf(phi0mesh, phi1mesh, loss_function, 256, cmap=my_colormap);\n",
+ " ax.contour(phi0mesh, phi1mesh, loss_function, 20, colors=['#80808080'])\n",
+ " ax.plot(opt_path[0,:], opt_path[1,:],'-', color='#a0d9d3ff')\n",
+ " ax.plot(opt_path[0,:], opt_path[1,:],'.', color='#a0d9d3ff',markersize=10)\n",
+ " ax.set_xlabel(\"$\\phi_{0}$\")\n",
+ " ax.set_ylabel(\"$\\phi_{1}$\")\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "YKijFyuH4ZJD"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Simple fixed step size gradient descent\n",
+ "def grad_descent(start_posn, n_steps, alpha):\n",
+ " grad_path = np.zeros((2, n_steps+1));\n",
+ " grad_path[:,0] = start_posn[:,0];\n",
+ " for c_step in range(n_steps):\n",
+ " this_grad = get_loss_gradient(grad_path[0,c_step], grad_path[1,c_step]);\n",
+ " grad_path[:,c_step+1] = grad_path[:,c_step] - alpha * this_grad\n",
+ " return grad_path;"
+ ],
+ "metadata": {
+ "id": "Afxr7RqR8s7Q"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "We'll start by running gradient descent with a fixed step size for this loss function."
+ ],
+ "metadata": {
+ "id": "MXZL8lu3-EUF"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "loss_function, phi0mesh, phi1mesh = get_loss_function_for_plot() ;\n",
+ "\n",
+ "start_posn = np.zeros((2,1));\n",
+ "start_posn[0,0] = -0.7; start_posn[1,0] = -0.9\n",
+ "\n",
+ "# Run gradient descent\n",
+ "grad_path1 = grad_descent(start_posn, n_steps=200, alpha = 0.08)\n",
+ "draw_function(phi0mesh, phi1mesh, loss_function, my_colormap, grad_path1)\n",
+ "grad_path2 = grad_descent(start_posn, n_steps=40, alpha= 1.0)\n",
+ "draw_function(phi0mesh, phi1mesh, loss_function, my_colormap, grad_path2)"
+ ],
+ "metadata": {
+ "id": "fgkwVEal8stH"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Because the function changes much faster in $\\phi_1$ than in $\\phi_0$, there is no great step size to choose. If we set the step size so that it makes sensible progress in the $\\phi_1$ direction, then it takes many iterations to converge. If we set the step size so that we make sensible progress in the $\\phi_0$ direction, then the path oscillates in the $\\phi_1$ direction. \n",
+ "\n",
+ "This motivates Adam. At the core of Adam is the idea that we should just determine which way is downhill along each axis (i.e. left/right for $\\phi_0$ or up/down for $\\phi_1$) and move a fixed distance in that direction."
+ ],
+ "metadata": {
+ "id": "AN2uNxaa-bRX"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def normalized_gradients(start_posn, n_steps, alpha, epsilon=1e-20):\n",
+ " grad_path = np.zeros((2, n_steps+1));\n",
+ " grad_path[:,0] = start_posn[:,0];\n",
+ " for c_step in range(n_steps):\n",
+ " # Measure the gradient as in equation 6.13 (first line)\n",
+ " m = get_loss_gradient(grad_path[0,c_step], grad_path[1,c_step]);\n",
+ " # TO DO -- compute the squared gradient as in equation 6.13 (second line)\n",
+ " # Replace this line:\n",
+ " v = np.ones_like(grad_path[:,0])\n",
+ "\n",
+ " # TO DO -- apply the update rule (equation 6.14)\n",
+ " # Replace this line:\n",
+ " grad_path[:,c_step+1] = grad_path[:,c_step]\n",
+ "\n",
+ " return grad_path;"
+ ],
+ "metadata": {
+ "id": "IqX2zP_29gLF"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's try out normalized gradients\n",
+ "start_posn = np.zeros((2,1));\n",
+ "start_posn[0,0] = -0.7; start_posn[1,0] = -0.9\n",
+ "\n",
+ "# Run gradient descent\n",
+ "grad_path1 = normalized_gradients(start_posn, n_steps=40, alpha = 0.08)\n",
+ "draw_function(phi0mesh, phi1mesh, loss_function, my_colormap, grad_path1)"
+ ],
+ "metadata": {
+ "id": "wxe-dKW5Chv3"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "This moves towards the minimum at a sensible speed, but we never actually converge -- the solution just bounces back and forth between the last two points. To make it converge, we add momentum to both the estimates of the gradient and the pointwise squared gradient. We also modify the statistics by a factor that depends on the time to make sure the progress is now slow to start with."
+ ],
+ "metadata": {
+ "id": "_6KoKBJdGGI4"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def adam(start_posn, n_steps, alpha, beta=0.9, gamma=0.99, epsilon=1e-20):\n",
+ " grad_path = np.zeros((2, n_steps+1));\n",
+ " grad_path[:,0] = start_posn[:,0];\n",
+ " m = np.zeros_like(grad_path[:,0])\n",
+ " v = np.zeros_like(grad_path[:,0])\n",
+ " for c_step in range(n_steps):\n",
+ " # Measure the gradient\n",
+ " grad = get_loss_gradient(grad_path[0,c_step], grad_path[1,c_step])\n",
+ " # TODO -- Update the momentum based gradient estimate equation 6.15 (first line)\n",
+ " # Replace this line:\n",
+ " m = m;\n",
+ "\n",
+ "\n",
+ " # TODO -- update the momentum based squared gradient estimate as in equation 6.15 (second line)\n",
+ " # Replace this line:\n",
+ " v = v\n",
+ "\n",
+ " # TODO -- Modify the statistics according to equation 6.16\n",
+ " # You will need the function np.power\n",
+ " # Replace these lines\n",
+ " m_tilde = m\n",
+ " v_tilde = v\n",
+ "\n",
+ "\n",
+ " # TO DO -- apply the update rule (equation 6.17)\n",
+ " # Replace this line:\n",
+ " grad_path[:,c_step+1] = grad_path[:,c_step]\n",
+ "\n",
+ " return grad_path;"
+ ],
+ "metadata": {
+ "id": "BKUhZSGgDEm0"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's try out our Adam algorithm\n",
+ "start_posn = np.zeros((2,1));\n",
+ "start_posn[0,0] = -0.7; start_posn[1,0] = -0.9\n",
+ "\n",
+ "# Run gradient descent\n",
+ "grad_path1 = adam(start_posn, n_steps=60, alpha = 0.05)\n",
+ "draw_function(phi0mesh, phi1mesh, loss_function, my_colormap, grad_path1)"
+ ],
+ "metadata": {
+ "id": "sg5X18P3IbYo"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap07/7_1_Backpropagation_in_Toy_Model.ipynb b/Notebooks/Chap07/7_1_Backpropagation_in_Toy_Model.ipynb
new file mode 100644
index 0000000..8b757f2
--- /dev/null
+++ b/Notebooks/Chap07/7_1_Backpropagation_in_Toy_Model.ipynb
@@ -0,0 +1,483 @@
+{
+ "cells": [
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "colab_type": "text",
+ "id": "view-in-github"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "pOZ6Djz0dhoy"
+ },
+ "source": [
+ "# **Notebook 7.1: Backpropagation in Toy Model**\n",
+ "\n",
+ "This notebook computes the derivatives of the toy function discussed in section 7.3 of the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "1DmMo2w63CmT"
+ },
+ "source": [
+ "We're going to investigate how to take the derivatives of functions where one operation is composed with another, which is composed with a third and so on. For example, consider the model:\n",
+ "\n",
+ "\\begin{equation}\n",
+ " \\text{f}[x,\\boldsymbol\\phi] = \\beta_3+\\omega_3\\cdot\\cos\\Bigl[\\beta_2+\\omega_2\\cdot\\exp\\bigl[\\beta_1+\\omega_1\\cdot\\sin[\\beta_0+\\omega_0x]\\bigr]\\Bigr],\n",
+ "\\end{equation}\n",
+ "\n",
+ "with parameters $\\boldsymbol\\phi=\\{\\beta_0,\\omega_0,\\beta_1,\\omega_1,\\beta_2,\\omega_2,\\beta_3,\\omega_3\\}$."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 7.2: Backpropagation**\n",
+ "\n",
+ "This notebook runs the backpropagation algorithm on a deep neural network as described in section 7.4 of the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "L6chybAVFJW2"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "LdIDglk1FFcG"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "First let's define a neural network. We'll just choose the weights and biases randomly for now"
+ ],
+ "metadata": {
+ "id": "nnUoI0m6GyjC"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set seed so we always get the same random numbers\n",
+ "np.random.seed(0)\n",
+ "\n",
+ "# Number of layers\n",
+ "K = 5\n",
+ "# Number of neurons per layer\n",
+ "D = 6\n",
+ "# Input layer\n",
+ "D_i = 1\n",
+ "# Output layer\n",
+ "D_o = 1\n",
+ "\n",
+ "# Make empty lists\n",
+ "all_weights = [None] * (K+1)\n",
+ "all_biases = [None] * (K+1)\n",
+ "\n",
+ "# Create input and output layers\n",
+ "all_weights[0] = np.random.normal(size=(D, D_i))\n",
+ "all_weights[-1] = np.random.normal(size=(D_o, D))\n",
+ "all_biases[0] = np.random.normal(size =(D,1))\n",
+ "all_biases[-1]= np.random.normal(size =(D_o,1))\n",
+ "\n",
+ "# Create intermediate layers\n",
+ "for layer in range(1,K):\n",
+ " all_weights[layer] = np.random.normal(size=(D,D))\n",
+ " all_biases[layer] = np.random.normal(size=(D,1))"
+ ],
+ "metadata": {
+ "id": "WVM4Tc_jGI0Q"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the Rectified Linear Unit (ReLU) function\n",
+ "def ReLU(preactivation):\n",
+ " activation = preactivation.clip(0.0)\n",
+ " return activation"
+ ],
+ "metadata": {
+ "id": "jZh-7bPXIDq4"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's run our random network. The weight matrices $\\boldsymbol\\Omega_{1\\ldots K}$ are the entries of the list \"all_weights\" and the biases $\\boldsymbol\\beta_{1\\ldots K}$ are the entries of the list \"all_biases\"\n",
+ "\n",
+ "We know that we will need the preactivations $\\mathbf{f}_{0\\ldots K}$ and the activations $\\mathbf{h}_{1\\ldots K}$ for the forward pass of backpropagation, so we'll store and return these as well.\n"
+ ],
+ "metadata": {
+ "id": "5irtyxnLJSGX"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def compute_network_output(net_input, all_weights, all_biases):\n",
+ "\n",
+ " # Retrieve number of layers\n",
+ " K = len(all_weights) -1\n",
+ "\n",
+ " # We'll store the pre-activations at each layer in a list \"all_f\"\n",
+ " # and the activations in a second list \"all_h\".\n",
+ " all_f = [None] * (K+1)\n",
+ " all_h = [None] * (K+1)\n",
+ "\n",
+ " #For convenience, we'll set\n",
+ " # all_h[0] to be the input, and all_f[K] will be the output\n",
+ " all_h[0] = net_input\n",
+ "\n",
+ " # Run through the layers, calculating all_f[0...K-1] and all_h[1...K]\n",
+ " for layer in range(K):\n",
+ " # Update preactivations and activations at this layer according to eqn 7.16\n",
+ " # Remmember to use np.matmul for matrix multiplications\n",
+ " # TODO -- Replace the lines below\n",
+ " all_f[layer] = all_h[layer]\n",
+ " all_h[layer+1] = all_f[layer]\n",
+ "\n",
+ " # Compute the output from the last hidden layer\n",
+ " # TO DO -- Replace the line below\n",
+ " all_f[K] = np.zeros_like(all_biases[-1])\n",
+ "\n",
+ " # Retrieve the output\n",
+ " net_output = all_f[K]\n",
+ "\n",
+ " return net_output, all_f, all_h"
+ ],
+ "metadata": {
+ "id": "LgquJUJvJPaN"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define input\n",
+ "net_input = np.ones((D_i,1)) * 1.2\n",
+ "# Compute network output\n",
+ "net_output, all_f, all_h = compute_network_output(net_input,all_weights, all_biases)\n",
+ "print(\"True output = %3.3f, Your answer = %3.3f\"%(1.907, net_output[0,0]))"
+ ],
+ "metadata": {
+ "id": "IN6w5m2ZOhnB"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's define a loss function. We'll just use the least squares loss function. We'll also write a function to compute dloss_doutput"
+ ],
+ "metadata": {
+ "id": "SxVTKp3IcoBF"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def least_squares_loss(net_output, y):\n",
+ " return np.sum((net_output-y) * (net_output-y))\n",
+ "\n",
+ "def d_loss_d_output(net_output, y):\n",
+ " return 2*(net_output -y);"
+ ],
+ "metadata": {
+ "id": "6XqWSYWJdhQR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "y = np.ones((D_o,1)) * 20.0\n",
+ "loss = least_squares_loss(net_output, y)\n",
+ "print(\"y = %3.3f Loss = %3.3f\"%(y, loss))"
+ ],
+ "metadata": {
+ "id": "njF2DUQmfttR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's compute the derivatives of the network. We already computed the forward pass. Let's compute the backward pass."
+ ],
+ "metadata": {
+ "id": "98WmyqFYWA-0"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# We'll need the indicator function\n",
+ "def indicator_function(x):\n",
+ " x_in = np.array(x)\n",
+ " x_in[x_in>=0] = 1\n",
+ " x_in[x_in<0] = 0\n",
+ " return x_in\n",
+ "\n",
+ "# Main backward pass routine\n",
+ "def backward_pass(all_weights, all_biases, all_f, all_h, y):\n",
+ " # We'll store the derivatives dl_dweights and dl_dbiases in lists as well\n",
+ " all_dl_dweights = [None] * (K+1)\n",
+ " all_dl_dbiases = [None] * (K+1)\n",
+ " # And we'll store the derivatives of the loss with respect to the activation and preactivations in lists\n",
+ " all_dl_df = [None] * (K+1)\n",
+ " all_dl_dh = [None] * (K+1)\n",
+ " # Again for convenience we'll stick with the convention that all_h[0] is the net input and all_f[k] in the net output\n",
+ "\n",
+ " # Compute derivatives of the loss with respect to the network output\n",
+ " all_dl_df[K] = np.array(d_loss_d_output(all_f[K],y))\n",
+ "\n",
+ " # Now work backwards through the network\n",
+ " for layer in range(K,-1,-1):\n",
+ " # TODO Calculate the derivatives of the loss with respect to the biases at layer from all_dl_df[layer]. (eq 7.21)\n",
+ " # NOTE! To take a copy of matrix X, use Z=np.array(X)\n",
+ " # REPLACE THIS LINE\n",
+ " all_dl_dbiases[layer] = np.zeros_like(all_biases[layer])\n",
+ "\n",
+ " # TODO Calculate the derivatives of the loss with respect to the weights at layer from all_dl_df[layer] and all_h[layer] (eq 7.22)\n",
+ " # Don't forget to use np.matmul\n",
+ " # REPLACE THIS LINE\n",
+ " all_dl_dweights[layer] = np.zeros_like(all_weights[layer])\n",
+ "\n",
+ " # TODO: calculate the derivatives of the loss with respect to the activations from weight and derivatives of next preactivations (second part of last line of eq 7.24)\n",
+ " # REPLACE THIS LINE\n",
+ " all_dl_dh[layer] = np.zeros_like(all_h[layer])\n",
+ "\n",
+ "\n",
+ " if layer > 0:\n",
+ " # TODO Calculate the derivatives of the loss with respect to the pre-activation f (use derivative of ReLu function, first part of last line of eq. 7.24)\n",
+ " # REPLACE THIS LINE\n",
+ " all_dl_df[layer-1] = np.zeros_like(all_f[layer-1])\n",
+ "\n",
+ " return all_dl_dweights, all_dl_dbiases"
+ ],
+ "metadata": {
+ "id": "LJng7WpRPLMz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "all_dl_dweights, all_dl_dbiases = backward_pass(all_weights, all_biases, all_f, all_h, y)"
+ ],
+ "metadata": {
+ "id": "9A9MHc4sQvbp"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "np.set_printoptions(precision=3)\n",
+ "# Make space for derivatives computed by finite differences\n",
+ "all_dl_dweights_fd = [None] * (K+1)\n",
+ "all_dl_dbiases_fd = [None] * (K+1)\n",
+ "\n",
+ "# Let's test if we have the derivatives right using finite differences\n",
+ "delta_fd = 0.000001\n",
+ "\n",
+ "# Test the dervatives of the bias vectors\n",
+ "for layer in range(K):\n",
+ " dl_dbias = np.zeros_like(all_dl_dbiases[layer])\n",
+ " # For every element in the bias\n",
+ " for row in range(all_biases[layer].shape[0]):\n",
+ " # Take copy of biases We'll change one element each time\n",
+ " all_biases_copy = [np.array(x) for x in all_biases]\n",
+ " all_biases_copy[layer][row] += delta_fd\n",
+ " network_output_1, *_ = compute_network_output(net_input, all_weights, all_biases_copy)\n",
+ " network_output_2, *_ = compute_network_output(net_input, all_weights, all_biases)\n",
+ " dl_dbias[row] = (least_squares_loss(network_output_1, y) - least_squares_loss(network_output_2,y))/delta_fd\n",
+ " all_dl_dbiases_fd[layer] = np.array(dl_dbias)\n",
+ " print(\"-----------------------------------------------\")\n",
+ " print(\"Bias %d, derivatives from backprop:\"%(layer))\n",
+ " print(all_dl_dbiases[layer])\n",
+ " print(\"Bias %d, derivatives from finite differences\"%(layer))\n",
+ " print(all_dl_dbiases_fd[layer])\n",
+ " if np.allclose(all_dl_dbiases_fd[layer],all_dl_dbiases[layer],rtol=1e-05, atol=1e-08, equal_nan=False):\n",
+ " print(\"Success! Derivatives match.\")\n",
+ " else:\n",
+ " print(\"Failure! Derivatives different.\")\n",
+ "\n",
+ "\n",
+ "\n",
+ "# Test the derivatives of the weights matrices\n",
+ "for layer in range(K):\n",
+ " dl_dweight = np.zeros_like(all_dl_dweights[layer])\n",
+ " # For every element in the bias\n",
+ " for row in range(all_weights[layer].shape[0]):\n",
+ " for col in range(all_weights[layer].shape[1]):\n",
+ " # Take copy of biases We'll change one element each time\n",
+ " all_weights_copy = [np.array(x) for x in all_weights]\n",
+ " all_weights_copy[layer][row][col] += delta_fd\n",
+ " network_output_1, *_ = compute_network_output(net_input, all_weights_copy, all_biases)\n",
+ " network_output_2, *_ = compute_network_output(net_input, all_weights, all_biases)\n",
+ " dl_dweight[row][col] = (least_squares_loss(network_output_1, y) - least_squares_loss(network_output_2,y))/delta_fd\n",
+ " all_dl_dweights_fd[layer] = np.array(dl_dweight)\n",
+ " print(\"-----------------------------------------------\")\n",
+ " print(\"Weight %d, derivatives from backprop:\"%(layer))\n",
+ " print(all_dl_dweights[layer])\n",
+ " print(\"Weight %d, derivatives from finite differences\"%(layer))\n",
+ " print(all_dl_dweights_fd[layer])\n",
+ " if np.allclose(all_dl_dweights_fd[layer],all_dl_dweights[layer],rtol=1e-05, atol=1e-08, equal_nan=False):\n",
+ " print(\"Success! Derivatives match.\")\n",
+ " else:\n",
+ " print(\"Failure! Derivatives different.\")"
+ ],
+ "metadata": {
+ "id": "PK-UtE3hreAK"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap07/7_3_Initialization.ipynb b/Notebooks/Chap07/7_3_Initialization.ipynb
new file mode 100644
index 0000000..673d273
--- /dev/null
+++ b/Notebooks/Chap07/7_3_Initialization.ipynb
@@ -0,0 +1,357 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyOaATWBrwVMylV1akcKtHjt",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 7.3: Initialization**\n",
+ "\n",
+ "This notebook explores weight initialization in deep neural networks as described in section 7.5 of the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "L6chybAVFJW2"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "LdIDglk1FFcG"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "First let's define a neural network. We'll just choose the weights and biases randomly for now"
+ ],
+ "metadata": {
+ "id": "nnUoI0m6GyjC"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def init_params(K, D, sigma_sq_omega):\n",
+ " # Set seed so we always get the same random numbers\n",
+ " np.random.seed(0)\n",
+ "\n",
+ " # Input layer\n",
+ " D_i = 1\n",
+ " # Output layer\n",
+ " D_o = 1\n",
+ "\n",
+ " # Make empty lists\n",
+ " all_weights = [None] * (K+1)\n",
+ " all_biases = [None] * (K+1)\n",
+ "\n",
+ " # Create input and output layers\n",
+ " all_weights[0] = np.random.normal(size=(D, D_i))*np.sqrt(sigma_sq_omega)\n",
+ " all_weights[-1] = np.random.normal(size=(D_o, D)) * np.sqrt(sigma_sq_omega)\n",
+ " all_biases[0] = np.zeros((D,1))\n",
+ " all_biases[-1]= np.zeros((D_o,1))\n",
+ "\n",
+ " # Create intermediate layers\n",
+ " for layer in range(1,K):\n",
+ " all_weights[layer] = np.random.normal(size=(D,D))*np.sqrt(sigma_sq_omega)\n",
+ " all_biases[layer] = np.zeros((D,1))\n",
+ "\n",
+ " return all_weights, all_biases"
+ ],
+ "metadata": {
+ "id": "WVM4Tc_jGI0Q"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the Rectified Linear Unit (ReLU) function\n",
+ "def ReLU(preactivation):\n",
+ " activation = preactivation.clip(0.0)\n",
+ " return activation"
+ ],
+ "metadata": {
+ "id": "jZh-7bPXIDq4"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def compute_network_output(net_input, all_weights, all_biases):\n",
+ "\n",
+ " # Retrieve number of layers\n",
+ " K = len(all_weights)-1\n",
+ "\n",
+ " # We'll store the pre-activations at each layer in a list \"all_f\"\n",
+ " # and the activations in a second list \"all_h\".\n",
+ " all_f = [None] * (K+1)\n",
+ " all_h = [None] * (K+1)\n",
+ "\n",
+ " #For convenience, we'll set\n",
+ " # all_h[0] to be the input, and all_f[K] will be the output\n",
+ " all_h[0] = net_input\n",
+ "\n",
+ " # Run through the layers, calculating all_f[0...K-1] and all_h[1...K]\n",
+ " for layer in range(K):\n",
+ " # Update preactivations and activations at this layer according to eqn 7.5\n",
+ " all_f[layer] = all_biases[layer] + np.matmul(all_weights[layer], all_h[layer])\n",
+ " all_h[layer+1] = ReLU(all_f[layer])\n",
+ "\n",
+ " # Compute the output from the last hidden layer\n",
+ " all_f[K] = all_biases[K] + np.matmul(all_weights[K], all_h[K])\n",
+ "\n",
+ " # Retrieve the output\n",
+ " net_output = all_f[K]\n",
+ "\n",
+ " return net_output, all_f, all_h"
+ ],
+ "metadata": {
+ "id": "LgquJUJvJPaN"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's investigate how the size of the outputs vary as we change the initialization variance:\n"
+ ],
+ "metadata": {
+ "id": "bIUrcXnOqChl"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Number of layers\n",
+ "K = 5\n",
+ "# Number of neurons per layer\n",
+ "D = 8\n",
+ "# Input layer\n",
+ "D_i = 1\n",
+ "# Output layer\n",
+ "D_o = 1\n",
+ "# Set variance of initial weights to 1\n",
+ "sigma_sq_omega = 1.0\n",
+ "# Initialize parameters\n",
+ "all_weights, all_biases = init_params(K,D,sigma_sq_omega)\n",
+ "\n",
+ "n_data = 1000\n",
+ "data_in = np.random.normal(size=(1,n_data))\n",
+ "net_output, all_f, all_h = compute_network_output(data_in, all_weights, all_biases)\n",
+ "\n",
+ "for layer in range(1,K+1):\n",
+ " print(\"Layer %d, std of hidden units = %3.3f\"%(layer, np.std(all_h[layer])))"
+ ],
+ "metadata": {
+ "id": "A55z3rKBqO7M"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# You can see that the values of the hidden units are increasing on average (the variance is across all hidden units at the layer\n",
+ "# and the 1000 training examples\n",
+ "\n",
+ "# TO DO\n",
+ "# Change this to 50 layers with 80 hidden units per layer\n",
+ "\n",
+ "# TO DO\n",
+ "# Now experiment with sigma_sq_omega to try to stop the variance of the forward computation exploding"
+ ],
+ "metadata": {
+ "id": "VL_SO4tar3DC"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's define a loss function. We'll just use the least squares loss function. We'll also write a function to compute dloss_doutput\n"
+ ],
+ "metadata": {
+ "id": "SxVTKp3IcoBF"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def least_squares_loss(net_output, y):\n",
+ " return np.sum((net_output-y) * (net_output-y))\n",
+ "\n",
+ "def d_loss_d_output(net_output, y):\n",
+ " return 2*(net_output -y);"
+ ],
+ "metadata": {
+ "id": "6XqWSYWJdhQR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Here's the code for the backward pass"
+ ],
+ "metadata": {
+ "id": "98WmyqFYWA-0"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# We'll need the indicator function\n",
+ "def indicator_function(x):\n",
+ " x_in = np.array(x)\n",
+ " x_in[x_in>=0] = 1\n",
+ " x_in[x_in<0] = 0\n",
+ " return x_in\n",
+ "\n",
+ "# Main backward pass routine\n",
+ "def backward_pass(all_weights, all_biases, all_f, all_h, y):\n",
+ " # Retrieve number of layers\n",
+ " K = all_weights\n",
+ "\n",
+ " # We'll store the derivatives dl_dweights and dl_dbiases in lists as well\n",
+ " all_dl_dweights = [None] * (K+1)\n",
+ " all_dl_dbiases = [None] * (K+1)\n",
+ " # And we'll store the derivatives of the loss with respect to the activation and preactivations in lists\n",
+ " all_dl_df = [None] * (K+1)\n",
+ " all_dl_dh = [None] * (K+1)\n",
+ " # Again for convenience we'll stick with the convention that all_h[0] is the net input and all_f[k] in the net output\n",
+ "\n",
+ " # Compute derivatives of net output with respect to loss\n",
+ " all_dl_df[K] = np.array(d_loss_d_output(all_f[K],y))\n",
+ "\n",
+ " # Now work backwards through the network\n",
+ " for layer in range(K,-1,-1):\n",
+ " # Calculate the derivatives of biases at layer from all_dl_df[K]. (eq 7.13, line 1)\n",
+ " all_dl_dbiases[layer] = np.array(all_dl_df[layer])\n",
+ " # Calculate the derivatives of weight at layer from all_dl_df[K] and all_h[K] (eq 7.13, line 2)\n",
+ " all_dl_dweights[layer] = np.matmul(all_dl_df[layer], all_h[layer].transpose())\n",
+ "\n",
+ " # Calculate the derivatives of activations from weight and derivatives of next preactivations (eq 7.13, line 3 second part)\n",
+ " all_dl_dh[layer] = np.matmul(all_weights[layer].transpose(), all_dl_df[layer])\n",
+ " # Calculate the derivatives of the pre-activation f with respect to activation h (eq 7.13, line 3, first part)\n",
+ " if layer > 0:\n",
+ " all_dl_df[layer-1] = indicator_function(all_f[layer-1]) * all_dl_dh[layer]\n",
+ "\n",
+ " return all_dl_dweights, all_dl_dbiases, all_dl_dh, all_dl_df"
+ ],
+ "metadata": {
+ "id": "LJng7WpRPLMz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's look at what happens to the magnitude of the gradients on the way back."
+ ],
+ "metadata": {
+ "id": "phFnbthqwhFi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Number of layers\n",
+ "K = 5\n",
+ "# Number of neurons per layer\n",
+ "D = 8\n",
+ "# Input layer\n",
+ "D_i = 1\n",
+ "# Output layer\n",
+ "D_o = 1\n",
+ "# Set variance of initial weights to 1\n",
+ "sigma_sq_omega = 1.0\n",
+ "# Initialize parameters\n",
+ "all_weights, all_biases = init_params(K,D,sigma_sq_omega)\n",
+ "\n",
+ "# For simplicity we'll just consider the gradients of the weights and biases between the first and last hidden layer\n",
+ "n_data = 100\n",
+ "aggregate_dl_df = [None] * (K+1)\n",
+ "for layer in range(1,K):\n",
+ " # These 3D arrays will store the gradients for every data point\n",
+ " aggregate_dl_df[layer] = np.zeros((D,n_data))\n",
+ "\n",
+ "\n",
+ "# We'll have to compute the derivatives of the parameters for each data point separately\n",
+ "for c_data in range(n_data):\n",
+ " data_in = np.random.normal(size=(1,1))\n",
+ " y = np.zeros((1,1))\n",
+ " net_output, all_f, all_h = compute_network_output(data_in, all_weights, all_biases)\n",
+ " all_dl_dweights, all_dl_dbiases, all_dl_dh, all_dl_df = backward_pass(all_weights, all_biases, all_f, all_h, y)\n",
+ " for layer in range(1,K):\n",
+ " aggregate_dl_df[layer][:,c_data] = np.squeeze(all_dl_df[layer])\n",
+ "\n",
+ "for layer in range(1,K):\n",
+ " print(\"Layer %d, std of dl_dh = %3.3f\"%(layer, np.std(aggregate_dl_df[layer].ravel())))\n"
+ ],
+ "metadata": {
+ "id": "9A9MHc4sQvbp"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# You can see that the values of the hidden units are increasing on average (the variance is across all hidden units at the layer\n",
+ "# and the 1000 training examples\n",
+ "\n",
+ "# TO DO\n",
+ "# Change this to 50 layers with 80 hidden units per layer\n",
+ "\n",
+ "# TO DO\n",
+ "# Now experiment with sigma_sq_omega to try to stop the variance of the gradients exploding\n"
+ ],
+ "metadata": {
+ "id": "gtokc0VX0839"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap08/8_1_MNIST_1D_Performance.ipynb b/Notebooks/Chap08/8_1_MNIST_1D_Performance.ipynb
new file mode 100644
index 0000000..0f8c1b3
--- /dev/null
+++ b/Notebooks/Chap08/8_1_MNIST_1D_Performance.ipynb
@@ -0,0 +1,238 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "gpuType": "T4",
+ "authorship_tag": "ABX9TyOuKMUcKfOIhIL2qTX9jJCy",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ },
+ "accelerator": "GPU"
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 8.1: MNIST_1D_Performance**\n",
+ "\n",
+ "This notebook runs a simple neural network on the MNIST1D dataset as in figure 8.2a. It uses code from https://github.com/greydanus/mnist1d to generate the data.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "L6chybAVFJW2"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
+ "!git clone https://github.com/greydanus/mnist1d"
+ ],
+ "metadata": {
+ "id": "ifVjS4cTOqKz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import torch, torch.nn as nn\n",
+ "from torch.utils.data import TensorDataset, DataLoader\n",
+ "from torch.optim.lr_scheduler import StepLR\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "import mnist1d"
+ ],
+ "metadata": {
+ "id": "qyE7G1StPIqO"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's generate a training and test dataset using the MNIST1D code. The dataset gets saved as a .pkl file so it doesn't have to be regenerated each time."
+ ],
+ "metadata": {
+ "id": "F7LNq72SP6jO"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "args = mnist1d.data.get_dataset_args()\n",
+ "data = mnist1d.data.get_dataset(args, path='./sample_data/mnist1d_data.pkl', download=False, regenerate=False)\n",
+ "\n",
+ "# The training and test input and outputs are in\n",
+ "# data['x'], data['y'], data['x_test'], and data['y_test']\n",
+ "print(\"Examples in training set: {}\".format(len(data['y'])))\n",
+ "print(\"Examples in test set: {}\".format(len(data['y_test'])))\n",
+ "print(\"Length of each example: {}\".format(data['x'].shape[-1]))"
+ ],
+ "metadata": {
+ "id": "YLxf7dJfPaqw"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "D_i = 40 # Input dimensions\n",
+ "D_k = 100 # Hidden dimensions\n",
+ "D_o = 10 # Output dimensions\n",
+ "# TO DO:\n",
+ "# Define a model with two hidden layers of size 100\n",
+ "# And ReLU activations between them\n",
+ "# Replace this line (see Figure 7.8 of book for help):\n",
+ "model = torch.nn.Sequential(torch.nn.Linear(D_i, D_o));\n",
+ "\n",
+ "\n",
+ "def weights_init(layer_in):\n",
+ " # TO DO:\n",
+ " # Initialize the parameters with He initialization\n",
+ " # Replace this line (see figure 7.8 of book for help)\n",
+ " print(\"Initializing layer\")\n",
+ "\n",
+ "\n",
+ "# Call the function you just defined\n",
+ "model.apply(weights_init)\n"
+ ],
+ "metadata": {
+ "id": "FxaB5vc0uevl"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# choose cross entropy loss function (equation 5.24)\n",
+ "loss_function = torch.nn.CrossEntropyLoss()\n",
+ "# construct SGD optimizer and initialize learning rate and momentum\n",
+ "optimizer = torch.optim.SGD(model.parameters(), lr = 0.05, momentum=0.9)\n",
+ "# object that decreases learning rate by half every 10 epochs\n",
+ "scheduler = StepLR(optimizer, step_size=10, gamma=0.5)\n",
+ "# create 100 dummy data points and store in data loader class\n",
+ "x_train = torch.tensor(data['x'].astype('float32'))\n",
+ "y_train = torch.tensor(data['y'].transpose().astype('long'))\n",
+ "x_test= torch.tensor(data['x_test'].astype('float32'))\n",
+ "y_test = torch.tensor(data['y_test'].astype('long'))\n",
+ "\n",
+ "# load the data into a class that creates the batches\n",
+ "data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
+ "\n",
+ "# Initialize model weights\n",
+ "model.apply(weights_init)\n",
+ "\n",
+ "# loop over the dataset n_epoch times\n",
+ "n_epoch = 50\n",
+ "# store the loss and the % correct at each epoch\n",
+ "losses_train = np.zeros((n_epoch))\n",
+ "errors_train = np.zeros((n_epoch))\n",
+ "losses_test = np.zeros((n_epoch))\n",
+ "errors_test = np.zeros((n_epoch))\n",
+ "\n",
+ "for epoch in range(n_epoch):\n",
+ " # loop over batches\n",
+ " for i, batch in enumerate(data_loader):\n",
+ " # retrieve inputs and labels for this batch\n",
+ " x_batch, y_batch = batch\n",
+ " # zero the parameter gradients\n",
+ " optimizer.zero_grad()\n",
+ " # forward pass -- calculate model output\n",
+ " pred = model(x_batch)\n",
+ " # compute the loss\n",
+ " loss = loss_function(pred, y_batch)\n",
+ " # backward pass\n",
+ " loss.backward()\n",
+ " # SGD update\n",
+ " optimizer.step()\n",
+ "\n",
+ " # Run whole dataset to get statistics -- normally wouldn't do this\n",
+ " pred_train = model(x_train)\n",
+ " pred_test = model(x_test)\n",
+ " _, predicted_train_class = torch.max(pred_train.data, 1)\n",
+ " _, predicted_test_class = torch.max(pred_test.data, 1)\n",
+ " errors_train[epoch] = 100 - 100 * (predicted_train_class == y_train).float().sum() / len(y_train)\n",
+ " errors_test[epoch]= 100 - 100 * (predicted_test_class == y_test).float().sum() / len(y_test)\n",
+ " losses_train[epoch] = loss_function(pred_train, y_train).item()\n",
+ " losses_test[epoch]= loss_function(pred_test, y_test).item()\n",
+ " print(f'Epoch {epoch:5d}, train loss {losses_train[epoch]:.6f}, train error {errors_train[epoch]:3.2f}, test loss {losses_test[epoch]:.6f}, test error {errors_test[epoch]:3.2f}')\n",
+ "\n",
+ " # tell scheduler to consider updating learning rate\n",
+ " scheduler.step()"
+ ],
+ "metadata": {
+ "id": "_rX6N3VyyQTY"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Plot the results\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(errors_train,'r-',label='train')\n",
+ "ax.plot(errors_test,'b-',label='test')\n",
+ "ax.set_ylim(0,100); ax.set_xlim(0,n_epoch)\n",
+ "ax.set_xlabel('Epoch'); ax.set_ylabel('Error')\n",
+ "ax.set_title('TrainError %3.2f, Test Error %3.2f'%(errors_train[-1],errors_test[-1]))\n",
+ "ax.legend()\n",
+ "plt.show()\n",
+ "\n",
+ "# Plot the results\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(losses_train,'r-',label='train')\n",
+ "ax.plot(losses_test,'b-',label='test')\n",
+ "ax.set_xlim(0,n_epoch)\n",
+ "ax.set_xlabel('Epoch'); ax.set_ylabel('Loss')\n",
+ "ax.set_title('Train loss %3.2f, Test loss %3.2f'%(losses_train[-1],losses_test[-1]))\n",
+ "ax.legend()\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "yI-l6kA_EH9G"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "**TO DO**\n",
+ "\n",
+ "Play with the model -- try changing the number of layers, hidden units, learning rate, batch size, momentum or anything else you like. See if you can improve the test results.\n",
+ "\n",
+ "Is it a good idea to optimize the hyperparameters in this way? Will the final result be a good estimate of the true test performance?"
+ ],
+ "metadata": {
+ "id": "q-yT6re6GZS4"
+ }
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap08/8_2_Bias_Variance_Trade_Off.ipynb b/Notebooks/Chap08/8_2_Bias_Variance_Trade_Off.ipynb
new file mode 100644
index 0000000..269f56c
--- /dev/null
+++ b/Notebooks/Chap08/8_2_Bias_Variance_Trade_Off.ipynb
@@ -0,0 +1,347 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 8.2: Bias-Variance Trade-Off**\n",
+ "\n",
+ "This notebook investigates the bias-variance trade-off for the toy model used throughout chapter 8 and reproduces the bias/variance trade off curves seen in figure 8.9.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "L6chybAVFJW2"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ],
+ "metadata": {
+ "id": "01Cu4SGZOVAi"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# The true function that we are trying to estimate, defined on [0,1]\n",
+ "def true_function(x):\n",
+ " y = np.exp(np.sin(x*(2*3.1413)))\n",
+ " return y"
+ ],
+ "metadata": {
+ "id": "bSK2_EGyOgHu"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Generate some data points with or without noise\n",
+ "def generate_data(n_data, sigma_y=0.3):\n",
+ " # Generate x values quasi uniformly\n",
+ " x = np.ones(n_data)\n",
+ " for i in range(n_data):\n",
+ " x[i] = np.random.uniform(i/n_data, (i+1)/n_data, 1)\n",
+ "\n",
+ " # y value from running through function and adding noise\n",
+ " y = np.ones(n_data)\n",
+ " for i in range(n_data):\n",
+ " y[i] = true_function(x[i])\n",
+ " y[i] += np.random.normal(0, sigma_y, 1)\n",
+ " return x,y\n"
+ ],
+ "metadata": {
+ "id": "yzZr2tcJO5pq"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Draw the fitted function, together win uncertainty used to generate points\n",
+ "def plot_function(x_func, y_func, x_data=None,y_data=None, x_model = None, y_model =None, sigma_func = None, sigma_model=None):\n",
+ "\n",
+ " fig,ax = plt.subplots()\n",
+ " ax.plot(x_func, y_func, 'k-')\n",
+ " if sigma_func is not None:\n",
+ " ax.fill_between(x_func, y_func-2*sigma_func, y_func+2*sigma_func, color='lightgray')\n",
+ "\n",
+ " if x_data is not None:\n",
+ " ax.plot(x_data, y_data, 'o', color='#d18362')\n",
+ "\n",
+ " if x_model is not None:\n",
+ " ax.plot(x_model, y_model, '-', color='#7fe7de')\n",
+ "\n",
+ " if sigma_model is not None:\n",
+ " ax.fill_between(x_model, y_model-2*sigma_model, y_model+2*sigma_model, color='lightgray')\n",
+ "\n",
+ " ax.set_xlim(0,1)\n",
+ " ax.set_xlabel('Input, $x$')\n",
+ " ax.set_ylabel('Output, $y$')\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "xfq1SD_ZOi6G"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Generate true function\n",
+ "x_func = np.linspace(0, 1.0, 100)\n",
+ "y_func = true_function(x_func);\n",
+ "\n",
+ "# Generate some data points\n",
+ "np.random.seed(1)\n",
+ "sigma_func = 0.3\n",
+ "n_data = 15\n",
+ "x_data,y_data = generate_data(n_data, sigma_func)\n",
+ "\n",
+ "# Plot the functinon, data and uncertainty\n",
+ "plot_function(x_func, y_func, x_data, y_data, sigma_func=sigma_func)"
+ ],
+ "metadata": {
+ "id": "2tP-p7B6Qnuf"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define model -- beta is a scalar and omega has size n_hidden,1\n",
+ "def network(x, beta, omega):\n",
+ " # Retrieve number of hidden units\n",
+ " n_hidden = omega.shape[0]\n",
+ "\n",
+ " y = np.zeros_like(x)\n",
+ " for c_hidden in range(n_hidden):\n",
+ " # Evaluate activations based on shifted lines (figure 8.4b-d)\n",
+ " line_vals = x - c_hidden/n_hidden\n",
+ " h = line_vals * (line_vals > 0)\n",
+ " # Weight activations by omega parameters and sum\n",
+ " y = y + omega[c_hidden] * h\n",
+ " # Add bias, beta\n",
+ " y = y + beta\n",
+ "\n",
+ " return y"
+ ],
+ "metadata": {
+ "id": "zYMLtS3nT-0y"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# This fits the n_hidden+1 parameters (see fig 8.4a) in closed form.\n",
+ "# If you have studied linear algebra, then you will know it is a least\n",
+ "# squares solution of the form (A^TA)^-1A^Tb. If you don't recognize that,\n",
+ "# then just take it on trust that this gives you the best possible solution.\n",
+ "def fit_model_closed_form(x,y,n_hidden):\n",
+ " n_data = len(x)\n",
+ " A = np.ones((n_data, n_hidden+1))\n",
+ " for i in range(n_data):\n",
+ " for j in range(1,n_hidden+1):\n",
+ " A[i,j] = x[i]-(j-1)/n_hidden\n",
+ " if A[i,j] < 0:\n",
+ " A[i,j] = 0;\n",
+ "\n",
+ " beta_omega = np.linalg.lstsq(A, y, rcond=None)[0]\n",
+ "\n",
+ " beta = beta_omega[0]\n",
+ " omega = beta_omega[1:]\n",
+ "\n",
+ " return beta, omega\n"
+ ],
+ "metadata": {
+ "id": "MinJxLh1XTHx"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Closed form solution\n",
+ "beta, omega = fit_model_closed_form(x_data,y_data,n_hidden=3)\n",
+ "\n",
+ "# Get prediction for model across graph grange\n",
+ "x_model = np.linspace(0,1,100);\n",
+ "y_model = network(x_model, beta, omega)\n",
+ "\n",
+ "# Draw the function and the model\n",
+ "plot_function(x_func, y_func, x_data,y_data, x_model, y_model)"
+ ],
+ "metadata": {
+ "id": "HP7fiwNFSfWz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run the model many times with different datasets and return the mean and variance\n",
+ "def get_model_mean_variance(n_data, n_datasets, n_hidden, sigma_func):\n",
+ "\n",
+ " # Create array that stores model results in rows\n",
+ " y_model_all = np.zeros((n_datasets, x_model.shape[0]))\n",
+ "\n",
+ " for c_dataset in range(n_datasets):\n",
+ " # TODO -- Generate n_data x,y, pairs with standard deviation sigma_func\n",
+ " # Replace this line\n",
+ " x_data,y_data = np.zeros([1,n_data]),np.zeros([1,n_data])\n",
+ "\n",
+ " # TODO -- Fit the model\n",
+ " # Replace this line:\n",
+ " beta = 0; omega = np.zeros([n_hidden,1])\n",
+ "\n",
+ " # TODO -- Run the fitted model on x_model\n",
+ " # Replace this line\n",
+ " y_model = np.zeros_like(x_model);\n",
+ "\n",
+ " # Store the model results\n",
+ " y_model_all[c_dataset,:] = y_model\n",
+ "\n",
+ " # Get mean and standard deviation of model\n",
+ " mean_model = np.mean(y_model_all,axis=0)\n",
+ " std_model = np.std(y_model_all,axis=0)\n",
+ "\n",
+ " # Return the mean and standard deviation of the fitted model\n",
+ " return mean_model, std_model"
+ ],
+ "metadata": {
+ "id": "bL553uSaYidy"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's generate N random data sets, fit the model N times and look the mean and variance\n",
+ "n_datasets = 100\n",
+ "n_data = 15\n",
+ "sigma_func = 0.3\n",
+ "n_hidden = 5\n",
+ "\n",
+ "# Get mean and variance of fitted model\n",
+ "np.random.seed(1)\n",
+ "mean_model, std_model = get_model_mean_variance(n_data, n_datasets, n_hidden, sigma_func) ;\n",
+ "\n",
+ "# Plot the results\n",
+ "plot_function(x_func, y_func, x_model=x_model, y_model=mean_model, sigma_model=std_model)"
+ ],
+ "metadata": {
+ "id": "Wxk64t2SoX9c"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO -- Experiment with changing the number of data points and the number of hidden variables\n",
+ "# in the model. Get a feeling for what happens in terms of the bias (squared deviation between cyan and black lines)\n",
+ "# and the variance (gray region) as we manipulate these quantities."
+ ],
+ "metadata": {
+ "id": "QO6mFaKNJ3J_"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Plot the noise, bias and variance as a function of capacity\n",
+ "hidden_variables = [1,2,3,4,5,6,7,8,9,10,11,12]\n",
+ "bias = np.zeros((len(hidden_variables),1)) ;\n",
+ "variance = np.zeros((len(hidden_variables),1)) ;\n",
+ "\n",
+ "n_datasets = 100\n",
+ "n_data = 15\n",
+ "sigma_func = 0.3\n",
+ "n_hidden = 5\n",
+ "\n",
+ "# Set random seed so that get same result every time\n",
+ "np.random.seed(1)\n",
+ "\n",
+ "for c_hidden in range(len(hidden_variables)):\n",
+ " # Get mean and variance of fitted model\n",
+ " mean_model, std_model = get_model_mean_variance(n_data, n_datasets, hidden_variables[c_hidden], sigma_func) ;\n",
+ " # TODO -- Estimate bias and variance\n",
+ " # Replace these lines\n",
+ "\n",
+ " # Compute variance -- average of the model variance (average squared deviation of fitted models around mean fitted model)\n",
+ " variance[c_hidden] = 0\n",
+ " # Compute bias (average squared deviation of mean fitted model around true function)\n",
+ " bias[c_hidden] = 0\n",
+ "\n",
+ "# Plot the results\n",
+ "fig,ax = plt.subplots()\n",
+ "ax.plot(hidden_variables, variance, 'k-')\n",
+ "ax.plot(hidden_variables, bias, 'r-')\n",
+ "ax.plot(hidden_variables, variance+bias, 'g-')\n",
+ "ax.set_xlim(0,12)\n",
+ "ax.set_ylim(0,0.5)\n",
+ "ax.set_xlabel(\"Model capacity\")\n",
+ "ax.set_ylabel(\"Variance\")\n",
+ "ax.legend(['Variance', 'Bias', 'Bias + Variance'])\n",
+ "plt.show()\n"
+ ],
+ "metadata": {
+ "id": "ICKjqAlx3Ka9"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "WKUyOAywL_b2"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap08/8_3_Double_Descent.ipynb b/Notebooks/Chap08/8_3_Double_Descent.ipynb
new file mode 100644
index 0000000..ee745d6
--- /dev/null
+++ b/Notebooks/Chap08/8_3_Double_Descent.ipynb
@@ -0,0 +1,270 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "gpuType": "T4",
+ "authorship_tag": "ABX9TyN/KUpEObCKnHZ/4Onp5sHG",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ },
+ "accelerator": "GPU"
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 8.3: Double Descent**\n",
+ "\n",
+ "This notebook investigates double descent as described in section 8.4 of the book.\n",
+ "\n",
+ "It uses the MNIST-1D database which can be found at https://github.com/greydanus/mnist1d\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "L6chybAVFJW2"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
+ "!git clone https://github.com/greydanus/mnist1d"
+ ],
+ "metadata": {
+ "id": "fn9BP5N5TguP"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import torch, torch.nn as nn\n",
+ "from torch.utils.data import TensorDataset, DataLoader\n",
+ "from torch.optim.lr_scheduler import StepLR\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "import mnist1d\n",
+ "import random\n",
+ "random.seed(0)\n",
+ "\n",
+ "# Try attaching to GPU -- Use \"Change Runtime Type to change to GPUT\"\n",
+ "DEVICE = str(torch.device('cuda' if torch.cuda.is_available() else 'cpu'))\n",
+ "print('Using:', DEVICE)"
+ ],
+ "metadata": {
+ "id": "hFxuHpRqTgri"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "args = mnist1d.data.get_dataset_args()\n",
+ "args.num_samples = 8000\n",
+ "args.train_split = 0.5\n",
+ "args.corr_noise_scale = 0.25\n",
+ "args.iid_noise_scale=2e-2\n",
+ "data = mnist1d.data.get_dataset(args, path='./mnist1d_data.pkl', download=False, regenerate=True)\n",
+ "\n",
+ "# Add 15% noise to training labels\n",
+ "for c_y in range(len(data['y'])):\n",
+ " random_number = random.random()\n",
+ " if random_number < 0.15 :\n",
+ " random_int = int(random.random() * 10)\n",
+ " data['y'][c_y] = random_int\n",
+ "\n",
+ "# The training and test input and outputs are in\n",
+ "# data['x'], data['y'], data['x_test'], and data['y_test']\n",
+ "print(\"Examples in training set: {}\".format(len(data['y'])))\n",
+ "print(\"Examples in test set: {}\".format(len(data['y_test'])))\n",
+ "print(\"Length of each example: {}\".format(data['x'].shape[-1]))"
+ ],
+ "metadata": {
+ "id": "PW2gyXL5UkLU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Initialize the parameters with He initialization\n",
+ "def weights_init(layer_in):\n",
+ " if isinstance(layer_in, nn.Linear):\n",
+ " nn.init.kaiming_uniform_(layer_in.weight)\n",
+ " layer_in.bias.data.fill_(0.0)\n",
+ "\n",
+ "# Return an initialized model with two hidden layers and n_hidden hidden units at each\n",
+ "def get_model(n_hidden):\n",
+ "\n",
+ " D_i = 40 # Input dimensions\n",
+ " D_k = n_hidden # Hidden dimensions\n",
+ " D_o = 10 # Output dimensions\n",
+ "\n",
+ " # Define a model with two hidden layers of size 100\n",
+ " # And ReLU activations between them\n",
+ " model = nn.Sequential(\n",
+ " nn.Linear(D_i, D_k),\n",
+ " nn.ReLU(),\n",
+ " nn.Linear(D_k, D_k),\n",
+ " nn.ReLU(),\n",
+ " nn.Linear(D_k, D_o))\n",
+ "\n",
+ " # Call the function you just defined\n",
+ " model.apply(weights_init)\n",
+ "\n",
+ " # Return the model\n",
+ " return model ;"
+ ],
+ "metadata": {
+ "id": "hAIvZOAlTnk9"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def fit_model(model, data):\n",
+ "\n",
+ " # choose cross entropy loss function (equation 5.24)\n",
+ " loss_function = torch.nn.CrossEntropyLoss()\n",
+ " # construct SGD optimizer and initialize learning rate and momentum\n",
+ " # optimizer = torch.optim.Adam(model.parameters(), lr=0.01)\n",
+ " optimizer = torch.optim.SGD(model.parameters(), lr = 0.01, momentum=0.9)\n",
+ "\n",
+ "\n",
+ " # create 100 dummy data points and store in data loader class\n",
+ " x_train = torch.tensor(data['x'].astype('float32'))\n",
+ " y_train = torch.tensor(data['y'].transpose().astype('long'))\n",
+ " x_test= torch.tensor(data['x_test'].astype('float32'))\n",
+ " y_test = torch.tensor(data['y_test'].astype('long'))\n",
+ "\n",
+ " # load the data into a class that creates the batches\n",
+ " data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
+ "\n",
+ " # loop over the dataset n_epoch times\n",
+ " n_epoch = 1000\n",
+ "\n",
+ " for epoch in range(n_epoch):\n",
+ " # loop over batches\n",
+ " for i, batch in enumerate(data_loader):\n",
+ " # retrieve inputs and labels for this batch\n",
+ " x_batch, y_batch = batch\n",
+ " # zero the parameter gradients\n",
+ " optimizer.zero_grad()\n",
+ " # forward pass -- calculate model output\n",
+ " pred = model(x_batch)\n",
+ " # compute the loss\n",
+ " loss = loss_function(pred, y_batch)\n",
+ " # backward pass\n",
+ " loss.backward()\n",
+ " # SGD update\n",
+ " optimizer.step()\n",
+ "\n",
+ " # Run whole dataset to get statistics -- normally wouldn't do this\n",
+ " pred_train = model(x_train)\n",
+ " pred_test = model(x_test)\n",
+ " _, predicted_train_class = torch.max(pred_train.data, 1)\n",
+ " _, predicted_test_class = torch.max(pred_test.data, 1)\n",
+ " errors_train = 100 - 100 * (predicted_train_class == y_train).float().sum() / len(y_train)\n",
+ " errors_test= 100 - 100 * (predicted_test_class == y_test).float().sum() / len(y_test)\n",
+ " losses_train = loss_function(pred_train, y_train).item()\n",
+ " losses_test= loss_function(pred_test, y_test).item()\n",
+ " if epoch%100 ==0 :\n",
+ " print(f'Epoch {epoch:5d}, train loss {losses_train:.6f}, train error {errors_train:3.2f}, test loss {losses_test:.6f}, test error {errors_test:3.2f}')\n",
+ "\n",
+ " return errors_train, errors_test\n"
+ ],
+ "metadata": {
+ "id": "AazlQhheWmHk"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The following code produces the double descent curve by training the model with different numbers of hidden units and plotting the test error.\n",
+ "\n",
+ "TO DO:\n",
+ "\n",
+ "*Before* you run the code, and considering that there are 4000 training examples predict:"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 8.4: High-dimensional spaces**\n",
+ "\n",
+ "This notebook investigates the strange properties of high-dimensional spaces as discussed in the notes at the end of chapter 8.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "EjLK-kA1KnYX"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4ESMmnkYEVAb"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "import scipy.special as sci"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# How close are points in high dimensions?\n",
+ "\n",
+ "In this part of the notebook, we investigate how close random points are in 2D, 100D, and 1000D. In each case, we generate 1000 points and calculate the Euclidean distance between each pair. "
+ ],
+ "metadata": {
+ "id": "MonbPEitLNgN"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Fix the random seed so we all have the same random numbers\n",
+ "np.random.seed(0)\n",
+ "n_data = 1000\n",
+ "# Create 1000 data examples (columns) each with 2 dimensions (rows)\n",
+ "n_dim = 2\n",
+ "x_2D = np.random.normal(size=(n_dim,n_data))\n",
+ "# Create 1000 data examples (columns) each with 100 dimensions (rows)\n",
+ "n_dim = 100\n",
+ "x_100D = np.random.normal(size=(n_dim,n_data))\n",
+ "# Create 1000 data examples (columns) each with 1000 dimensions (rows)\n",
+ "n_dim = 1000\n",
+ "x_1000D = np.random.normal(size=(n_dim,n_data))"
+ ],
+ "metadata": {
+ "id": "vZSHVmcWEk14"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def distance_ratio(x):\n",
+ " # TODO -- replace the two lines below to calculate the largest and smallest Euclidean distance between\n",
+ " # the data points in the columns of x. DO NOT include the distance between the data point\n",
+ " # and itself (which is obviously zero)\n",
+ " smallest_dist = 1.0\n",
+ " largest_dist = 1.0\n",
+ "\n",
+ " # Calculate the ratio and return\n",
+ " dist_ratio = largest_dist / smallest_dist\n",
+ " return dist_ratio"
+ ],
+ "metadata": {
+ "id": "PhVmnUs8ErD9"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "print('Ratio of largest to smallest distance 2D: %3.3f'%(distance_ratio(x_2D)))\n",
+ "print('Ratio of largest to smallest distance 100D: %3.3f'%(distance_ratio(x_100D)))\n",
+ "print('Ratio of largest to smallest distance 1000D: %3.3f'%(distance_ratio(x_1000D)))\n"
+ ],
+ "metadata": {
+ "id": "0NdPxfn5GQuJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "If you did this right, you will see that the distance between the nearest and farthest two points in high dimensions is almost the same. "
+ ],
+ "metadata": {
+ "id": "uT68c0k8uBxs"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Volume of a hypersphere\n",
+ "\n",
+ "In the second part of this notebook we calculate the volume of a hypersphere of radius 0.5 (i.e., of diameter 1) as a function of the radius. Note that you you can check your answer by doing the calculation for 2D using the standard formula for the area of a circle and making sure it matches."
+ ],
+ "metadata": {
+ "id": "b2FYKV1SL4Z7"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def volume_of_hypersphere(diameter, dimensions):\n",
+ " # Formula given in Problem 8.7 of the book\n",
+ " # You will need sci.gamma()\n",
+ " # Check out: https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.gamma.html\n",
+ " # Also use this value for pi\n",
+ " pi = np.pi\n",
+ " # TODO replace this code with formula for the volume of a hypersphere\n",
+ " volume = 1.0\n",
+ "\n",
+ " return volume\n"
+ ],
+ "metadata": {
+ "id": "CZoNhD8XJaHR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "diameter = 1.0\n",
+ "for c_dim in range(1,11):\n",
+ " print(\"Volume of unit diameter hypersphere in %d dimensions is %3.3f\"%(c_dim, volume_of_hypersphere(diameter, c_dim)))"
+ ],
+ "metadata": {
+ "id": "fNTBlg_GPEUh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "You should see that the volume decreases to almost nothing in high dimensions. All of the volume is in the corners of the unit hypercube (which always has volume 1)."
+ ],
+ "metadata": {
+ "id": "PtaeGSNBunJl"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Proportion of hypersphere in outer shell\n",
+ "\n",
+ "In the third part of the notebook you will calculate what proportion of the volume of a hypersphere is in the outer 1% of the radius/diameter. Calculate the volume of a hypersphere and then the volume of a hypersphere with 0.99 of the radius and then figure out the ratio. "
+ ],
+ "metadata": {
+ "id": "GdyMeOBmoXyF"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def get_prop_of_volume_in_outer_1_percent(dimension):\n",
+ " # TODO -- replace this line\n",
+ " proportion = 1.0\n",
+ "\n",
+ " return proportion"
+ ],
+ "metadata": {
+ "id": "8_CxZ2AIpQ8w"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# While we're here, let's look at how much of the volume is in the outer 1% of the radius\n",
+ "for c_dim in [1,2,10,20,50,100,150,200,250,300]:\n",
+ " print('Proportion of volume in outer 1 percent of radius in %d dimensions =%3.3f'%(c_dim, get_prop_of_volume_in_outer_1_percent(c_dim)))"
+ ],
+ "metadata": {
+ "id": "LtMDIn2qPVfJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "You should see see that by the time we get to 300 dimensions most of the volume is in the outer 1 percent. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 9.1: L2 Regularization**\n",
+ "\n",
+ "This notebook investigates adding L2 regularization to the loss function for the Gabor model as in figure 9.1.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
+ ],
+ "metadata": {
+ "id": "el8l05WQEO46"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "xhmIOLiZELV_"
+ },
+ "outputs": [],
+ "source": [
+ "# import libraries\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from matplotlib import cm\n",
+ "from matplotlib.colors import ListedColormap"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's create our training data 30 pairs {x_i, y_i}\n",
+ "# We'll try to fit the Gabor model to these data\n",
+ "data = np.array([[-1.920e+00,-1.422e+01,1.490e+00,-1.940e+00,-2.389e+00,-5.090e+00,\n",
+ " -8.861e+00,3.578e+00,-6.010e+00,-6.995e+00,3.634e+00,8.743e-01,\n",
+ " -1.096e+01,4.073e-01,-9.467e+00,8.560e+00,1.062e+01,-1.729e-01,\n",
+ " 1.040e+01,-1.261e+01,1.574e-01,-1.304e+01,-2.156e+00,-1.210e+01,\n",
+ " -1.119e+01,2.902e+00,-8.220e+00,-1.179e+01,-8.391e+00,-4.505e+00],\n",
+ " [-1.051e+00,-2.482e-02,8.896e-01,-4.943e-01,-9.371e-01,4.306e-01,\n",
+ " 9.577e-03,-7.944e-02 ,1.624e-01,-2.682e-01,-3.129e-01,8.303e-01,\n",
+ " -2.365e-02,5.098e-01,-2.777e-01,3.367e-01,1.927e-01,-2.222e-01,\n",
+ " 6.352e-02,6.888e-03,3.224e-02,1.091e-02,-5.706e-01,-5.258e-02,\n",
+ " -3.666e-02,1.709e-01,-4.805e-02,2.008e-01,-1.904e-01,5.952e-01]])"
+ ],
+ "metadata": {
+ "id": "4cRkrh9MZ58Z"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Gabor model definition\n",
+ "def model(phi,x):\n",
+ " sin_component = np.sin(phi[0] + 0.06 * phi[1] * x)\n",
+ " gauss_component = np.exp(-(phi[0] + 0.06 * phi[1] * x) * (phi[0] + 0.06 * phi[1] * x) / 32)\n",
+ " y_pred= sin_component * gauss_component\n",
+ " return y_pred"
+ ],
+ "metadata": {
+ "id": "WQUERmb2erAe"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Draw model\n",
+ "def draw_model(data,model,phi,title=None):\n",
+ " x_model = np.arange(-15,15,0.1)\n",
+ " y_model = model(phi,x_model)\n",
+ "\n",
+ " fix, ax = plt.subplots()\n",
+ " ax.plot(data[0,:],data[1,:],'bo')\n",
+ " ax.plot(x_model,y_model,'m-')\n",
+ " ax.set_xlim([-15,15]);ax.set_ylim([-1,1])\n",
+ " ax.set_xlabel('x'); ax.set_ylabel('y')\n",
+ " if title is not None:\n",
+ " ax.set_title(title)\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "qFRe9POHF2le"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Initialize the parameters and draw the model\n",
+ "phi = np.zeros((2,1))\n",
+ "phi[0] = -5 # Horizontal offset\n",
+ "phi[1] = 25 # Frequency\n",
+ "draw_model(data,model,phi, \"Initial parameters\")\n"
+ ],
+ "metadata": {
+ "id": "TXx1Tpd1Tl-I"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's\n",
+ "compute the sum of squares loss for the training data"
+ ],
+ "metadata": {
+ "id": "QU5mdGvpTtEG"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def compute_loss(data_x, data_y, model, phi):\n",
+ " pred_y = model(phi, data_x)\n",
+ " loss = np.sum((pred_y-data_y)*(pred_y-data_y))\n",
+ " return loss"
+ ],
+ "metadata": {
+ "id": "I7dqTY2Gg7CR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's plot the whole loss function"
+ ],
+ "metadata": {
+ "id": "F3trnavPiHpH"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define pretty colormap\n",
+ "my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ "my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ "r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ "g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ "b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ "my_colormap = ListedColormap(np.vstack((r,g,b)).transpose()/255.0)\n",
+ "\n",
+ "def draw_loss_function(compute_loss, data, model, my_colormap, phi_iters = None):\n",
+ "\n",
+ " # Make grid of intercept/slope values to plot\n",
+ " offsets_mesh, freqs_mesh = np.meshgrid(np.arange(-10,10.0,0.1), np.arange(2.5,22.5,0.1))\n",
+ " loss_mesh = np.zeros_like(freqs_mesh)\n",
+ " # Compute loss for every set of parameters\n",
+ " for idslope, slope in np.ndenumerate(freqs_mesh):\n",
+ " loss_mesh[idslope] = compute_loss(data[0,:], data[1,:], model, np.array([[offsets_mesh[idslope]], [slope]]))\n",
+ "\n",
+ " fig,ax = plt.subplots()\n",
+ " fig.set_size_inches(8,8)\n",
+ " ax.contourf(offsets_mesh,freqs_mesh,loss_mesh,256,cmap=my_colormap)\n",
+ " ax.contour(offsets_mesh,freqs_mesh,loss_mesh,20,colors=['#80808080'])\n",
+ " if phi_iters is not None:\n",
+ " ax.plot(phi_iters[0,:], phi_iters[1,:],'go-')\n",
+ " ax.set_ylim([2.5,22.5])\n",
+ " ax.set_xlabel('Offset $\\phi_{0}$'); ax.set_ylabel('Frequency, $\\phi_{1}$')\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "K-NTHpAAHlCl"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "draw_loss_function(compute_loss, data, model, my_colormap)"
+ ],
+ "metadata": {
+ "id": "l8HbvIupnTME"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's compute the gradient vector for a given set of parameters:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\frac{\\partial L}{\\partial \\boldsymbol\\phi} = \\begin{bmatrix}\\frac{\\partial L}{\\partial \\phi_0} \\\\\\frac{\\partial L}{\\partial \\phi_1} \\end{bmatrix}.\n",
+ "\\end{equation}"
+ ],
+ "metadata": {
+ "id": "s9Duf05WqqSC"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# These came from writing out the expression for the sum of squares loss and taking the\n",
+ "# derivative with respect to phi0 and phi1. It was a lot of hassle to get it right!\n",
+ "def gabor_deriv_phi0(data_x,data_y,phi0, phi1):\n",
+ " x = 0.06 * phi1 * data_x + phi0\n",
+ " y = data_y\n",
+ " cos_component = np.cos(x)\n",
+ " sin_component = np.sin(x)\n",
+ " gauss_component = np.exp(-0.5 * x *x / 16)\n",
+ " deriv = cos_component * gauss_component - sin_component * gauss_component * x / 16\n",
+ " deriv = 2* deriv * (sin_component * gauss_component - y)\n",
+ " return np.sum(deriv)\n",
+ "\n",
+ "def gabor_deriv_phi1(data_x, data_y,phi0, phi1):\n",
+ " x = 0.06 * phi1 * data_x + phi0\n",
+ " y = data_y\n",
+ " cos_component = np.cos(x)\n",
+ " sin_component = np.sin(x)\n",
+ " gauss_component = np.exp(-0.5 * x *x / 16)\n",
+ " deriv = 0.06 * data_x * cos_component * gauss_component - 0.06 * data_x*sin_component * gauss_component * x / 16\n",
+ " deriv = 2*deriv * (sin_component * gauss_component - y)\n",
+ " return np.sum(deriv)\n",
+ "\n",
+ "def compute_gradient(data_x, data_y, phi):\n",
+ " dl_dphi0 = gabor_deriv_phi0(data_x, data_y, phi[0],phi[1])\n",
+ " dl_dphi1 = gabor_deriv_phi1(data_x, data_y, phi[0],phi[1])\n",
+ " # Return the gradient\n",
+ " return np.array([[dl_dphi0],[dl_dphi1]])"
+ ],
+ "metadata": {
+ "id": "UpswmkL2qwBT"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now we are ready to find the minimum. For simplicity, we'll just use regular (non-stochastic) gradient descent with a fixed learning rate."
+ ],
+ "metadata": {
+ "id": "5EIjMM9Fw2eT"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def gradient_descent_step(phi, data, model):\n",
+ " # Step 1: Compute the gradient\n",
+ " gradient = compute_gradient(data[0,:],data[1,:], phi)\n",
+ " # Step 2: Update the parameters -- note we want to search in the negative (downhill direction)\n",
+ " alpha = 0.1\n",
+ " phi = phi - alpha * gradient\n",
+ " return phi"
+ ],
+ "metadata": {
+ "id": "YVq6rmaWRD2M"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Initialize the parameters\n",
+ "n_steps = 41\n",
+ "phi_all = np.zeros((2,n_steps+1))\n",
+ "phi_all[0,0] = 2.6\n",
+ "phi_all[1,0] = 8.5\n",
+ "\n",
+ "# Measure loss and draw initial model\n",
+ "loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,0:1])\n",
+ "draw_model(data,model,phi_all[:,0:1], \"Initial parameters, Loss = %f\"%(loss))\n",
+ "\n",
+ "for c_step in range (n_steps):\n",
+ " # Do gradient descent step\n",
+ " phi_all[:,c_step+1:c_step+2] = gradient_descent_step(phi_all[:,c_step:c_step+1],data, model)\n",
+ " # Measure loss and draw model every 4th step\n",
+ " if c_step % 8 == 0:\n",
+ " loss = compute_loss(data[0,:], data[1,:], model, phi_all[:,c_step+1:c_step+2])\n",
+ " draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
+ "\n",
+ "draw_loss_function(compute_loss, data, model, my_colormap, phi_all)\n"
+ ],
+ "metadata": {
+ "id": "tOLd0gtdRLLS"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Unfortunately, when we start from this position, the solution descends to a local minimum and the final model doesn't fit well."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 9.2: Implicit Regularization**\n",
+ "\n",
+ "This notebook investigates how the finite step sizes in gradient descent cause the trajectory to deviate and how this can be explained by adding an implicit regularization term. It recreates figure 9.3 from the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
+ ],
+ "metadata": {
+ "id": "el8l05WQEO46"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "xhmIOLiZELV_"
+ },
+ "outputs": [],
+ "source": [
+ "# import libraries\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from matplotlib.colors import ListedColormap"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "#Create colormap\n",
+ "my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ "my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ "r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ "g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ "b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ "my_colormap_vals = np.vstack((r,g,b)).transpose()/255.0\n",
+ "my_colormap = ListedColormap(my_colormap_vals)"
+ ],
+ "metadata": {
+ "id": "RSrv2Ab7hbsY"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# define main function\n",
+ "def loss(phi0, phi1):\n",
+ " phi1_std = np.exp(-0.5 * (phi0 * phi0)*4.0)\n",
+ " return 1.0-np.exp(-0.5 * (phi1 * phi1)/(phi1_std * phi1_std))\n",
+ "\n",
+ "# Compute the gradient (just done with finite differences for simplicity)\n",
+ "def get_loss_gradient(phi0, phi1):\n",
+ " delta_phi = 0.00001;\n",
+ " gradient = np.zeros((2,1));\n",
+ " gradient[0] = (loss(phi0+delta_phi/2.0, phi1) - loss(phi0-delta_phi/2.0, phi1))/delta_phi\n",
+ " gradient[1] = (loss(phi0, phi1+delta_phi/2.0) - loss(phi0, phi1-delta_phi/2.0))/delta_phi\n",
+ " return gradient;"
+ ],
+ "metadata": {
+ "id": "Wzi8xmOmhkGF"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# define grid to plot function\n",
+ "grid_values = np.arange(-0.8,0.5,0.01);\n",
+ "phi0mesh, phi1mesh = np.meshgrid(grid_values, grid_values)\n",
+ "loss_function = np.zeros((grid_values.size, grid_values.size))\n",
+ "for idphi0, phi0 in enumerate(grid_values):\n",
+ " for idphi1, phi1 in enumerate(grid_values):\n",
+ " loss_function[idphi0, idphi1] = loss(phi1,phi0)\n"
+ ],
+ "metadata": {
+ "id": "SFJgLbH5iAZe"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Perform gradient descent n_steps times and return path\n",
+ "def grad_descent(start_posn, n_steps, step_size):\n",
+ " grad_path = np.zeros((2, n_steps+1));\n",
+ " grad_path[:,0] = start_posn[:,0];\n",
+ " for c_step in range(n_steps):\n",
+ " this_grad = get_loss_gradient(grad_path[0,c_step], grad_path[1,c_step]);\n",
+ " grad_path[:,c_step+1] = grad_path[:,c_step] - step_size * this_grad[:,0]\n",
+ " return grad_path;"
+ ],
+ "metadata": {
+ "id": "1BKt38kOh5TU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Draw the loss function and the trajectories on it\n",
+ "def draw_function(phi0mesh, phi1mesh, loss_function, my_colormap, grad_path_tiny_lr=None, grad_path_typical_lr=None):\n",
+ " fig = plt.figure();\n",
+ " ax = plt.axes();\n",
+ " fig.set_size_inches(7,7)\n",
+ " ax.contourf(phi0mesh, phi1mesh, loss_function, 256, cmap=my_colormap);\n",
+ " ax.contour(phi0mesh, phi1mesh, loss_function, 20, colors=['#80808080'])\n",
+ " ax.set_xlabel('$\\phi_{0}$'); ax.set_ylabel('$\\phi_{1}$')\n",
+ "\n",
+ " if grad_path_typical_lr is not None:\n",
+ " ax.plot(grad_path_typical_lr[0,:], grad_path_typical_lr[1,:],'ro-')\n",
+ " if grad_path_tiny_lr is not None:\n",
+ " ax.plot(grad_path_tiny_lr[0,:], grad_path_tiny_lr[1,:],'b-')\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "KMoEZIiEiQ33"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the start position\n",
+ "start_posn = np.zeros((2,1)); start_posn[0,0] = -0.7; start_posn[1,0] = -0.75\n",
+ "\n",
+ "# Run the gradient descent with a very small learning rate to simulate continuous case\n",
+ "grad_path_tiny_lr = grad_descent(start_posn, 10000, 0.001)\n",
+ "# Run the gradient descent with a typical sized learning rate\n",
+ "grad_path_typical_lr = grad_descent(start_posn, 100, 0.05)\n",
+ "\n",
+ "draw_function(phi0mesh, phi1mesh, loss_function, my_colormap, grad_path_tiny_lr, grad_path_typical_lr)\n"
+ ],
+ "metadata": {
+ "id": "FYpJ3cB4iy_B"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "You can see that the two solutions do not converge to the same place. The ideal continuous solution is in blue, but in practice, we run the gradient set with as large a learning rate as possible so that it converges quickly (red curve). "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 9.3: Ensembling**\n",
+ "\n",
+ "This notebook investigates how ensembling can improve the performance of models. We'll work with the simplified neural network model (figure 8.4 of book) which we can fit in closed form, and so we can eliminate any errors due to not finding the global maximum.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
+ ],
+ "metadata": {
+ "id": "el8l05WQEO46"
+ }
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "xhmIOLiZELV_"
+ },
+ "outputs": [],
+ "source": [
+ "# import libraries\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "# Define seed so get same results each time\n",
+ "np.random.seed(1)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# The true function that we are trying to estimate, defined on [0,1]\n",
+ "def true_function(x):\n",
+ " y = np.exp(np.sin(x*(2*3.1413)))\n",
+ " return y"
+ ],
+ "metadata": {
+ "id": "3hpqmFyQNrbt"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Generate some data points with or without noise\n",
+ "def generate_data(n_data, sigma_y=0.3):\n",
+ " # Generate x values quasi uniformly\n",
+ " x = np.ones(n_data)\n",
+ " for i in range(n_data):\n",
+ " x[i] = np.random.uniform(i/n_data, (i+1)/n_data, 1)\n",
+ "\n",
+ " # y value from running through functoin and adding noise\n",
+ " y = np.ones(n_data)\n",
+ " for i in range(n_data):\n",
+ " y[i] = true_function(x[i])\n",
+ " y[i] += np.random.normal(0, sigma_y, 1)\n",
+ " return x,y"
+ ],
+ "metadata": {
+ "id": "skZMM5TbNwq4"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Draw the fitted function, together win uncertainty used to generate points\n",
+ "def plot_function(x_func, y_func, x_data=None,y_data=None, x_model = None, y_model =None, sigma_func = None, sigma_model=None):\n",
+ "\n",
+ " fig,ax = plt.subplots()\n",
+ " ax.plot(x_func, y_func, 'k-')\n",
+ " if sigma_func is not None:\n",
+ " ax.fill_between(x_func, y_func-2*sigma_func, y_func+2*sigma_func, color='lightgray')\n",
+ "\n",
+ " if x_data is not None:\n",
+ " ax.plot(x_data, y_data, 'o', color='#d18362')\n",
+ "\n",
+ " if x_model is not None:\n",
+ " ax.plot(x_model, y_model, '-', color='#7fe7de')\n",
+ "\n",
+ " if sigma_model is not None:\n",
+ " ax.fill_between(x_model, y_model-2*sigma_model, y_model+2*sigma_model, color='lightgray')\n",
+ "\n",
+ " ax.set_xlim(0,1)\n",
+ " ax.set_xlabel('Input, $x$')\n",
+ " ax.set_ylabel('Output, $y$')\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "ziwD_R7lN0DY"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Generate true function\n",
+ "x_func = np.linspace(0, 1.0, 100)\n",
+ "y_func = true_function(x_func);\n",
+ "\n",
+ "# Generate some data points\n",
+ "np.random.seed(1)\n",
+ "sigma_func = 0.3\n",
+ "n_data = 15\n",
+ "x_data,y_data = generate_data(n_data, sigma_func)\n",
+ "\n",
+ "# Plot the functinon, data and uncertainty\n",
+ "plot_function(x_func, y_func, x_data, y_data, sigma_func=sigma_func)"
+ ],
+ "metadata": {
+ "id": "2CgKanwaN3NM"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define model -- beta is a scalar and omega has size n_hidden,1\n",
+ "def network(x, beta, omega):\n",
+ " # Retrieve number of hidden units\n",
+ " n_hidden = omega.shape[0]\n",
+ "\n",
+ " y = np.zeros_like(x)\n",
+ " for c_hidden in range(n_hidden):\n",
+ " # Evaluate activations based on shifted lines (figure 8.4b-d)\n",
+ " line_vals = x - c_hidden/n_hidden\n",
+ " h = line_vals * (line_vals > 0)\n",
+ " # Weight activations by omega parameters and sum\n",
+ " y = y + omega[c_hidden] * h\n",
+ " # Add bias, beta\n",
+ " y = y + beta\n",
+ "\n",
+ " return y"
+ ],
+ "metadata": {
+ "id": "gorZ6i97N7AR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# This fits the n_hidden+1 parameters (see fig 8.4a) in closed form.\n",
+ "# If you have studied linear algebra, then you will know it is a least\n",
+ "# squares solution of the form (A^TA)^-1A^Tb. If you don't recognize that,\n",
+ "# then just take it on trust that this gives you the best possible solution.\n",
+ "def fit_model_closed_form(x,y,n_hidden):\n",
+ " n_data = len(x)\n",
+ " A = np.ones((n_data, n_hidden+1))\n",
+ " for i in range(n_data):\n",
+ " for j in range(1,n_hidden+1):\n",
+ " # Compute preactivation\n",
+ " A[i,j] = x[i]-(j-1)/n_hidden\n",
+ " # Apply the ReLU function\n",
+ " if A[i,j] < 0:\n",
+ " A[i,j] = 0;\n",
+ "\n",
+ " # Add a tiny bit of regularization\n",
+ " reg_value = 0.00001\n",
+ " regMat = reg_value * np.identity(n_hidden+1)\n",
+ " regMat[0,0] = 0\n",
+ "\n",
+ " ATA = np.matmul(np.transpose(A), A) +regMat\n",
+ " ATAInv = np.linalg.inv(ATA)\n",
+ " ATAInvAT = np.matmul(ATAInv, np.transpose(A))\n",
+ " beta_omega = np.matmul(ATAInvAT,y)\n",
+ " beta = beta_omega[0]\n",
+ " omega = beta_omega[1:]\n",
+ "\n",
+ " return beta, omega"
+ ],
+ "metadata": {
+ "id": "bMrLZUIqOwiM"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Closed form solution\n",
+ "beta, omega = fit_model_closed_form(x_data,y_data,n_hidden=14)\n",
+ "\n",
+ "# Get prediction for model across graph grange\n",
+ "x_model = np.linspace(0,1,100);\n",
+ "y_model = network(x_model, beta, omega)\n",
+ "\n",
+ "# Draw the function and the model\n",
+ "plot_function(x_func, y_func, x_data,y_data, x_model, y_model)\n",
+ "\n",
+ "# Compute the mean squared error between the fitted model (cyan) and the true curve (black)\n",
+ "mean_sq_error = np.mean((y_model-y_func) * (y_model-y_func))\n",
+ "print(f\"Mean square error = {mean_sq_error:3.3f}\")"
+ ],
+ "metadata": {
+ "id": "mzmtdY8DOz16"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now let's resample the data with replacement four times.\n",
+ "n_model = 4\n",
+ "# Array to store the prediction from all of our models\n",
+ "all_y_model = np.zeros((n_model, len(y_model)))\n",
+ "\n",
+ "# For each model\n",
+ "for c_model in range(n_model):\n",
+ " # TODO Sample data indices with replacement (use np.random.choice)\n",
+ " # Replace this line\n",
+ " resampled_indices = np.arange(0,n_data,1);\n",
+ "\n",
+ " # Extract the resampled x and y data\n",
+ " x_data_resampled = x_data[resampled_indices]\n",
+ " y_data_resampled = y_data[resampled_indices]\n",
+ "\n",
+ " # Fit the model\n",
+ " beta, omega = fit_model_closed_form(x_data_resampled,y_data_resampled,n_hidden=14)\n",
+ "\n",
+ " # Run the model\n",
+ " y_model_resampled = network(x_model, beta, omega)\n",
+ "\n",
+ " # Store the results\n",
+ " all_y_model[c_model,:] = y_model_resampled\n",
+ "\n",
+ " # Draw the function and the model\n",
+ " plot_function(x_func, y_func, x_data,y_data, x_model, y_model_resampled)\n",
+ "\n",
+ " # Compute the mean squared error between the fitted model (cyan) and the true curve (black)\n",
+ " mean_sq_error = np.mean((y_model_resampled-y_func) * (y_model_resampled-y_func))\n",
+ " print(f\"Mean square error = {mean_sq_error:3.3f}\")"
+ ],
+ "metadata": {
+ "id": "UKrAOEiKO8Go"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Plot the median of the results\n",
+ "# TODO -- find the median prediction\n",
+ "# Replace this line\n",
+ "y_model_median = all_y_model[0,:]\n",
+ "\n",
+ "# Draw the function and the model\n",
+ "plot_function(x_func, y_func, x_data,y_data, x_model, y_model_median)\n",
+ "\n",
+ "# Compute the mean squared error between the fitted model (cyan) and the true curve (black)\n",
+ "mean_sq_error = np.mean((y_model_median-y_func) * (y_model_median-y_func))\n",
+ "print(f\"Mean square error = {mean_sq_error:3.3f}\")"
+ ],
+ "metadata": {
+ "id": "cUTaW_GMS6nb"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Plot the median of the results\n",
+ "# TODO -- find the mean prediction\n",
+ "# Replace this line\n",
+ "y_model_mean = all_y_model[0,:]\n",
+ "\n",
+ "# Draw the function and the model\n",
+ "plot_function(x_func, y_func, x_data,y_data, x_model, y_model_mean)\n",
+ "\n",
+ "# Compute the mean squared error between the fitted model (cyan) and the true curve (black)\n",
+ "mean_sq_error = np.mean((y_model_mean-y_func) * (y_model_mean-y_func))\n",
+ "print(f\"Mean square error = {mean_sq_error:3.3f}\")"
+ ],
+ "metadata": {
+ "id": "2MKxwVxuRvCx"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "You should see that both the median and mean models are better than any of the individual models. We have improved our performance at the cost of four times as much training time, storage, and inference time."
+ ],
+ "metadata": {
+ "id": "K-jDZrfjWwBa"
+ }
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap09/9_4_Bayesian_Approach.ipynb b/Notebooks/Chap09/9_4_Bayesian_Approach.ipynb
new file mode 100644
index 0000000..25eac22
--- /dev/null
+++ b/Notebooks/Chap09/9_4_Bayesian_Approach.ipynb
@@ -0,0 +1,436 @@
+{
+ "cells": [
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "colab_type": "text",
+ "id": "view-in-github"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "el8l05WQEO46"
+ },
+ "source": [
+ "# **Notebook 9.4: Bayesian approach**\n",
+ "\n",
+ "This notebook investigates the Bayesian approach to model fitting and reproduces figure 9.11 from the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "xhmIOLiZELV_"
+ },
+ "outputs": [],
+ "source": [
+ "# import libraries\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "# Define seed so get same results each time\n",
+ "np.random.seed(1)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "3hpqmFyQNrbt"
+ },
+ "outputs": [],
+ "source": [
+ "# The true function that we are trying to estimate, defined on [0,1]\n",
+ "def true_function(x):\n",
+ " y = np.exp(np.sin(x*(2*3.1413)))\n",
+ " return y"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "skZMM5TbNwq4"
+ },
+ "outputs": [],
+ "source": [
+ "# Generate some data points with or without noise\n",
+ "def generate_data(n_data, sigma_y=0.3):\n",
+ " # Generate x values quasi uniformly\n",
+ " x = np.ones(n_data)\n",
+ " for i in range(n_data):\n",
+ " x[i] = np.random.uniform(i/n_data, (i+1)/n_data, 1)\n",
+ "\n",
+ " # y value from running through function and adding noise\n",
+ " y = np.ones(n_data)\n",
+ " for i in range(n_data):\n",
+ " y[i] = true_function(x[i])\n",
+ " y[i] += np.random.normal(0, sigma_y, 1)\n",
+ " return x,y"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "ziwD_R7lN0DY"
+ },
+ "outputs": [],
+ "source": [
+ "# Draw the fitted function, together win uncertainty used to generate points\n",
+ "def plot_function(x_func, y_func, x_data=None,y_data=None, x_model = None, y_model =None, sigma_func = None, sigma_model=None):\n",
+ "\n",
+ " fig,ax = plt.subplots()\n",
+ " ax.plot(x_func, y_func, 'k-')\n",
+ " if sigma_func is not None:\n",
+ " ax.fill_between(x_func, y_func-2*sigma_func, y_func+2*sigma_func, color='lightgray')\n",
+ "\n",
+ " if x_data is not None:\n",
+ " ax.plot(x_data, y_data, 'o', color='#d18362')\n",
+ "\n",
+ " if x_model is not None:\n",
+ " ax.plot(x_model, y_model, '-', color='#7fe7de')\n",
+ "\n",
+ " if sigma_model is not None:\n",
+ " ax.fill_between(x_model, y_model-2*sigma_model, y_model+2*sigma_model, color='lightgray')\n",
+ "\n",
+ " ax.set_xlim(0,1)\n",
+ " ax.set_xlabel('Input, $x$')\n",
+ " ax.set_ylabel('Output, $y$')\n",
+ " plt.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "2CgKanwaN3NM"
+ },
+ "outputs": [],
+ "source": [
+ "# Generate true function\n",
+ "x_func = np.linspace(0, 1.0, 100)\n",
+ "y_func = true_function(x_func);\n",
+ "\n",
+ "# Generate some data points\n",
+ "np.random.seed(1)\n",
+ "sigma_func = 0.3\n",
+ "n_data = 15\n",
+ "x_data,y_data = generate_data(n_data, sigma_func)\n",
+ "\n",
+ "# Plot the function, data and uncertainty\n",
+ "plot_function(x_func, y_func, x_data, y_data, sigma_func=sigma_func)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "gorZ6i97N7AR"
+ },
+ "outputs": [],
+ "source": [
+ "# Define model -- beta is a scalar and omega has size n_hidden,1\n",
+ "def network(x, beta, omega):\n",
+ " # Retrieve number of hidden units\n",
+ " n_hidden = omega.shape[0]\n",
+ "\n",
+ " y = np.zeros_like(x)\n",
+ " for c_hidden in range(n_hidden):\n",
+ " # Evaluate activations based on shifted lines (figure 8.4b-d)\n",
+ " line_vals = x - c_hidden/n_hidden\n",
+ " h = line_vals * (line_vals > 0)\n",
+ " # Weight activations by omega parameters and sum\n",
+ " y = y + omega[c_hidden] * h\n",
+ " # Add bias, beta\n",
+ " y = y + beta\n",
+ "\n",
+ " return y"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "i8T_QduzeBmM"
+ },
+ "source": [
+ "Now let's compute a probability distribution over the model parameters using Bayes's rule:\n",
+ "\n",
+ "\\begin{equation}\n",
+ " Pr(\\boldsymbol\\phi|\\{\\mathbf{x}_{i},\\mathbf{y}_{i}\\}) = \\frac{\\prod_{i=1}^{I} Pr(\\mathbf{y}_{i}|\\mathbf{x}_{i},\\boldsymbol\\phi) Pr(\\boldsymbol\\phi)}{\\int \\prod_{i=1}^{I} Pr(\\mathbf{y}_{i}|\\mathbf{x}_{i},\\boldsymbol\\phi) Pr(\\boldsymbol\\phi)d\\boldsymbol\\phi } ,\n",
+ "\\end{equation}\n",
+ "\n",
+ "We'll define the prior $Pr(\\boldsymbol\\phi)$ as:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "Pr(\\boldsymbol\\phi) = \\text{Norm}_{\\boldsymbol\\phi}\\bigl[\\mathbf{0},\\sigma^2_p\\mathbf{I}\\bigr]\n",
+ "\\end{equation}\n",
+ "\n",
+ "where $\\phi=[\\omega_1,\\omega_2\\ldots \\omega_n, \\beta]^T$ and $\\sigma^2_{p}$ is the prior variance.\n",
+ "\n",
+ "The likelihood term $\\prod_{i=1}^{I} Pr(\\mathbf{y}_{i}|\\mathbf{x}_{i},\\boldsymbol\\phi)$ is given by:\n",
+ "\n",
+ "\\begin{align}\n",
+ "\\prod_{i=1}^{I} Pr(\\mathbf{y}_{i}|\\mathbf{x}_{i},\\boldsymbol\\phi) &=& \\prod_{i=1}^{I} \\text{Norm}_{y_i}\\bigl[\\text{f}[\\mathbf{x}_{i},\\boldsymbol\\phi],\\sigma_d^2\\bigr]\\\\\n",
+ "&=& \\prod_{i=1}^{I} \\text{Norm}_{y_i}\\bigl[\\boldsymbol\\omega\\mathbf{h}_i+\\beta,\\sigma_d^2\\bigr]\\\\\n",
+ "&=& \\text{Norm}_{\\mathbf{y}}\\bigl[\\mathbf{H}^T\\boldsymbol\\phi,\\sigma^2\\mathbf{I}\\bigr].\n",
+ "\\end{align}\n",
+ "\n",
+ "where $\\sigma^2$ is the measurement noise and $\\mathbf{h}_{i}$ is the column vector of hidden variables for the $i^{th}$ input. Here the vector $\\mathbf{y}$ and matrix $\\mathbf{H}$ are defined as:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\mathbf{y} = \\begin{bmatrix}y_1\\\\y_2\\\\\\vdots\\\\y_{I}\\end{bmatrix}\\quad\\text{and}\\quad \\mathbf{H} = \\begin{bmatrix}\\mathbf{h}_{1}&\\mathbf{h}_{2}&\\cdots&\\mathbf{h}_{I}\\\\1&1&\\cdots &1\\end{bmatrix}.\n",
+ "\\end{equation}\n"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "JojV6ueRk49G"
+ },
+ "source": [
+ "To make progress we use the change of variable relation (Appendix C.3.4 of the book) to rewrite the likelihood term as a normal distribution in the parameters $\\boldsymbol\\phi$:\n",
+ "\n",
+ "\\begin{align}\n",
+ "\\prod_{i=1}^{I} Pr(\\mathbf{y}_{i}|\\mathbf{x}_{i},\\boldsymbol\\phi+\\beta)\n",
+ "&=& \\text{Norm}_{\\mathbf{y}}\\bigl[\\mathbf{H}^T\\boldsymbol\\phi,\\sigma^2\\bigr]\\\\\n",
+ "&\\propto& \\text{Norm}_{\\boldsymbol\\phi}\\bigl[(\\mathbf{H}\\mathbf{H}^T)^{-1}\\mathbf{H}\\mathbf{y},\\sigma^2(\\mathbf{H}\\mathbf{H}^t)^{-1}\\bigr]\n",
+ "\\end{align}\n"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "YX0O_Ciwp4W1"
+ },
+ "source": [
+ "Finally, we can combine the likelihood and prior terms using the product of two normal distributions relation (Appendix C.3.3).\n",
+ "\n",
+ "\\begin{align}\n",
+ " Pr(\\boldsymbol\\phi|\\{\\mathbf{x}_{i},\\mathbf{y}_{i}\\}) &\\propto& \\prod_{i=1}^{I} Pr(\\mathbf{y}_{i}|\\mathbf{x}_{i},\\boldsymbol\\phi) Pr(\\boldsymbol\\phi)\\\\\n",
+ " &\\propto&\\text{Norm}_{\\boldsymbol\\phi}\\bigl[(\\mathbf{H}\\mathbf{H}^T)^{-1}\\mathbf{H}\\mathbf{y},\\sigma^2(\\mathbf{H}\\mathbf{H}^T)^{-1}\\bigr] \\text{Norm}_{\\boldsymbol\\phi}\\bigl[\\mathbf{0},\\sigma^2_p\\mathbf{I}\\bigr]\\\\\n",
+ " &\\propto&\\text{Norm}_{\\boldsymbol\\phi}\\biggl[\\frac{1}{\\sigma^2}\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\mathbf{H}\\mathbf{y},\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\biggr].\n",
+ "\\end{align}\n",
+ "\n",
+ "In fact, since this already a normal distribution, the constant of proportionality must be one and we can write\n",
+ "\n",
+ "\\begin{align}\n",
+ " Pr(\\boldsymbol\\phi|\\{\\mathbf{x}_{i},\\mathbf{y}_{i}\\}) &=& \\text{Norm}_{\\boldsymbol\\phi}\\biggl[\\frac{1}{\\sigma^2}\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\mathbf{H}\\mathbf{y},\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\biggr].\n",
+ "\\end{align}\n",
+ "\n",
+ "TODO -- On a piece of paper, use the relations in Appendix C.3.3 and C.3.4 to fill in the missing steps and establish that this is the correct formula for the posterior."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "nF1AcgNDwm4t"
+ },
+ "outputs": [],
+ "source": [
+ "def compute_H(x_data, n_hidden):\n",
+ " psi1 = np.ones((n_hidden+1,1));\n",
+ " psi0 = np.linspace(0.0, 1.0, num=n_hidden, endpoint=False) * -1\n",
+ "\n",
+ " n_data = x_data.size\n",
+ " # First compute the hidden variables\n",
+ " H = np.ones((n_hidden+1, n_data))\n",
+ " for i in range(n_hidden):\n",
+ " for j in range(n_data):\n",
+ " # Compute preactivation\n",
+ " H[i,j] = psi1[i] * x_data[j]+psi0[i]\n",
+ " # Apply ReLU to get activation\n",
+ " if H[i,j] < 0:\n",
+ " H[i,j] = 0;\n",
+ "\n",
+ " return H\n",
+ "\n",
+ "def compute_param_mean_covar(x_data, y_data, n_hidden, sigma_sq, sigma_p_sq):\n",
+ " # Retrieve the matrix containing the hidden variables\n",
+ " H = compute_H(x_data, n_hidden) ;\n",
+ "\n",
+ " # TODO -- Compute the covariance matrix (you will need np.transpose(), np.matmul(), np.linalg.inv())\n",
+ " # Replace this line\n",
+ " phi_covar = np.identity(n_hidden+1)\n",
+ "\n",
+ "\n",
+ " # TODO -- Compute the mean matrix\n",
+ " # Replace this line\n",
+ " phi_mean = np.zeros((n_hidden+1,1))\n",
+ "\n",
+ "\n",
+ " return phi_mean, phi_covar"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "GjPnlG4q0UFK"
+ },
+ "source": [
+ "Now we can draw samples from this distribution"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "K4vYc82D0BMq"
+ },
+ "outputs": [],
+ "source": [
+ "# Define parameters\n",
+ "n_hidden = 5\n",
+ "sigma_sq = sigma_func * sigma_func\n",
+ "# Arbitrary large value reflecting the fact we are uncertain about the\n",
+ "# parameters before we see any data\n",
+ "sigma_p_sq = 1000\n",
+ "\n",
+ "# Compute the mean and covariance matrix\n",
+ "phi_mean, phi_covar = compute_param_mean_covar(x_data, y_data, n_hidden, sigma_sq, sigma_p_sq)\n",
+ "\n",
+ "# Let's draw the mean model\n",
+ "x_model = x_func\n",
+ "y_model_mean = network(x_model, phi_mean[-1], phi_mean[0:n_hidden])\n",
+ "plot_function(x_func, y_func, x_data, y_data, x_model, y_model_mean)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "TVIjhubkSw-R"
+ },
+ "outputs": [],
+ "source": [
+ "# TODO Draw two samples from the normal distribution over the parameters\n",
+ "# Replace these lines\n",
+ "phi_sample1 = np.zeros((n_hidden+1,1))\n",
+ "phi_sample2 = np.zeros((n_hidden+1,1))\n",
+ "\n",
+ "\n",
+ "# Run the network for these two sample sets of parameters\n",
+ "y_model_sample1 = network(x_model, phi_sample1[-1], phi_sample1[0:n_hidden])\n",
+ "y_model_sample2 = network(x_model, phi_sample2[-1], phi_sample2[0:n_hidden])\n",
+ "\n",
+ "# Draw the two models\n",
+ "plot_function(x_func, y_func, x_data, y_data, x_model, y_model_sample1)\n",
+ "plot_function(x_func, y_func, x_data, y_data, x_model, y_model_sample2)"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "GiNg5EroUiUb"
+ },
+ "source": [
+ "Now we need to perform inference for a new data points $\\mathbf{x}^*$ with corresponding hidden values $\\mathbf{h}^*$. Instead of having a single estimate of the parameters, we have a distribution over the possible parameters. So we marginalize (integrate) over this distribution to account for all possible values:\n",
+ "\n",
+ "\\begin{align}\n",
+ "Pr(y^*|\\mathbf{x}^*) &=& \\int Pr(y^{*}|\\mathbf{x}^*,\\boldsymbol\\phi)Pr(\\boldsymbol\\phi|\\{\\mathbf{x}_{i},\\mathbf{y}_{i}\\}) d\\boldsymbol\\phi\\\\\n",
+ "&=& \\int \\text{Norm}_{y^*}\\bigl[[\\mathbf{h}^{*T},1]\\boldsymbol\\phi,\\sigma^2\\bigr]\\cdot\\text{Norm}_{\\boldsymbol\\phi}\\biggl[\\frac{1}{\\sigma^2}\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\mathbf{H}\\mathbf{y},\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\biggr]d\\boldsymbol\\phi\\\\\n",
+ "&=& \\text{Norm}_{y^*}\\biggl[\\frac{1}{\\sigma^2} [\\mathbf{h}^{*T},1]\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\mathbf{H}\\mathbf{y}, [\\mathbf{h}^{*T},1]\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\n",
+ "[\\mathbf{h}^*;1]\\biggr]\n",
+ "\\end{align}\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "To compute this, we reformulated the integrand using the relations from appendices\n",
+ "C.3.3 and C.3.4 as the product of a normal distribution in $\\boldsymbol\\phi$ and a constant with respect\n",
+ "to $\\boldsymbol\\phi$. The integral of the normal distribution must be one, and so the final result is just the constant. This constant is itself a normal distribution in $y^*$. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 9.5: Augmentation**\n",
+ "\n",
+ "This notebook investigates data augmentation for the MNIST-1D model.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
+ ],
+ "metadata": {
+ "id": "el8l05WQEO46"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
+ "!git clone https://github.com/greydanus/mnist1d"
+ ],
+ "metadata": {
+ "id": "syvgxgRr3myY"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import torch, torch.nn as nn\n",
+ "from torch.utils.data import TensorDataset, DataLoader\n",
+ "from torch.optim.lr_scheduler import StepLR\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "import mnist1d\n",
+ "import random"
+ ],
+ "metadata": {
+ "id": "ckrNsYd13pMe"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "args = mnist1d.data.get_dataset_args()\n",
+ "data = mnist1d.data.get_dataset(args, path='./mnist1d_data.pkl', download=False, regenerate=False)\n",
+ "\n",
+ "# The training and test input and outputs are in\n",
+ "# data['x'], data['y'], data['x_test'], and data['y_test']\n",
+ "print(\"Examples in training set: {}\".format(len(data['y'])))\n",
+ "print(\"Examples in test set: {}\".format(len(data['y_test'])))\n",
+ "print(\"Length of each example: {}\".format(data['x'].shape[-1]))"
+ ],
+ "metadata": {
+ "id": "D_Woo9U730lZ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "D_i = 40 # Input dimensions\n",
+ "D_k = 200 # Hidden dimensions\n",
+ "D_o = 10 # Output dimensions\n",
+ "\n",
+ "# Define a model with two hidden layers of size 100\n",
+ "# And ReLU activations between them\n",
+ "model = nn.Sequential(\n",
+ "nn.Linear(D_i, D_k),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(D_k, D_k),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(D_k, D_o))\n",
+ "\n",
+ "def weights_init(layer_in):\n",
+ " # Initialize the parameters with He initialization\n",
+ " if isinstance(layer_in, nn.Linear):\n",
+ " nn.init.kaiming_uniform_(layer_in.weight)\n",
+ " layer_in.bias.data.fill_(0.0)\n",
+ "\n",
+ "# Call the function you just defined\n",
+ "model.apply(weights_init)"
+ ],
+ "metadata": {
+ "id": "JfIFWFIL33eF"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# choose cross entropy loss function (equation 5.24)\n",
+ "loss_function = torch.nn.CrossEntropyLoss()\n",
+ "# construct SGD optimizer and initialize learning rate and momentum\n",
+ "optimizer = torch.optim.SGD(model.parameters(), lr = 0.05, momentum=0.9)\n",
+ "# object that decreases learning rate by half every 10 epochs\n",
+ "scheduler = StepLR(optimizer, step_size=10, gamma=0.5)\n",
+ "# create 100 dummy data points and store in data loader class\n",
+ "x_train = torch.tensor(data['x'].astype('float32'))\n",
+ "y_train = torch.tensor(data['y'].transpose().astype('long'))\n",
+ "x_test= torch.tensor(data['x_test'].astype('float32'))\n",
+ "y_test = torch.tensor(data['y_test'].astype('long'))\n",
+ "\n",
+ "# load the data into a class that creates the batches\n",
+ "data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
+ "\n",
+ "# Initialize model weights\n",
+ "model.apply(weights_init)\n",
+ "\n",
+ "# loop over the dataset n_epoch times\n",
+ "n_epoch = 50\n",
+ "# store the loss and the % correct at each epoch\n",
+ "errors_train = np.zeros((n_epoch))\n",
+ "errors_test = np.zeros((n_epoch))\n",
+ "\n",
+ "for epoch in range(n_epoch):\n",
+ " # loop over batches\n",
+ " for i, batch in enumerate(data_loader):\n",
+ " # retrieve inputs and labels for this batch\n",
+ " x_batch, y_batch = batch\n",
+ " # zero the parameter gradients\n",
+ " optimizer.zero_grad()\n",
+ " # forward pass -- calculate model output\n",
+ " pred = model(x_batch)\n",
+ " # compute the loss\n",
+ " loss = loss_function(pred, y_batch)\n",
+ " # backward pass\n",
+ " loss.backward()\n",
+ " # SGD update\n",
+ " optimizer.step()\n",
+ "\n",
+ " # Run whole dataset to get statistics -- normally wouldn't do this\n",
+ " pred_train = model(x_train)\n",
+ " pred_test = model(x_test)\n",
+ " _, predicted_train_class = torch.max(pred_train.data, 1)\n",
+ " _, predicted_test_class = torch.max(pred_test.data, 1)\n",
+ " errors_train[epoch] = 100 - 100 * (predicted_train_class == y_train).float().sum() / len(y_train)\n",
+ " errors_test[epoch]= 100 - 100 * (predicted_test_class == y_test).float().sum() / len(y_test)\n",
+ " print(f'Epoch {epoch:5d}, train error {errors_train[epoch]:3.2f}, test error {errors_test[epoch]:3.2f}')"
+ ],
+ "metadata": {
+ "id": "YFfVbTPE4BkJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Plot the results\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(errors_train,'r-',label='train')\n",
+ "ax.plot(errors_test,'b-',label='test')\n",
+ "ax.set_ylim(0,100); ax.set_xlim(0,n_epoch)\n",
+ "ax.set_xlabel('Epoch'); ax.set_ylabel('Error')\n",
+ "ax.set_title('TrainError %3.2f, Test Error %3.2f'%(errors_train[-1],errors_test[-1]))\n",
+ "ax.legend()\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "FmGDd4vB8LyM"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The best test performance is about 33%. Let's see if we can improve on that by augmenting the data."
+ ],
+ "metadata": {
+ "id": "55XvoPDO8Qp-"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def augment(input_vector):\n",
+ " # Create output vector\n",
+ " data_out = np.zeros_like(input_vector)\n",
+ "\n",
+ " # TODO: Shift the input data by a random offset\n",
+ " # (rotating, so points that would go off the end, are added back to the beginning)\n",
+ " # Replace this line:\n",
+ " data_out = np.zeros_like(input_vector) ;\n",
+ "\n",
+ " # TODO: # Randomly scale the data by a factor drawn from a uniform distribution over [0.8,1.2]\n",
+ " # Replace this line:\n",
+ " data_out = np.array(data_out)\n",
+ "\n",
+ " return data_out"
+ ],
+ "metadata": {
+ "id": "IP6z2iox8MOF"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "n_data_orig = data['x'].shape[0]\n",
+ "# We'll double the amount o fdata\n",
+ "n_data_augment = n_data_orig+4000\n",
+ "augmented_x = np.zeros((n_data_augment, D_i))\n",
+ "augmented_y = np.zeros(n_data_augment)\n",
+ "# First n_data_orig rows are original data\n",
+ "augmented_x[0:n_data_orig,:] = data['x']\n",
+ "augmented_y[0:n_data_orig] = data['y']\n",
+ "\n",
+ "# Fill in rest of with augmented data\n",
+ "for c_augment in range(n_data_orig, n_data_augment):\n",
+ " # Choose a data point randomly\n",
+ " random_data_index = random.randint(0, n_data_orig-1)\n",
+ " # Augment the point and store\n",
+ " augmented_x[c_augment,:] = augment(data['x'][random_data_index,:])\n",
+ " augmented_y[c_augment] = data['y'][random_data_index]\n"
+ ],
+ "metadata": {
+ "id": "bzN0lu5J95AJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# choose cross entropy loss function (equation 5.24)\n",
+ "loss_function = torch.nn.CrossEntropyLoss()\n",
+ "# construct SGD optimizer and initialize learning rate and momentum\n",
+ "optimizer = torch.optim.SGD(model.parameters(), lr = 0.05, momentum=0.9)\n",
+ "# object that decreases learning rate by half every 50 epochs\n",
+ "scheduler = StepLR(optimizer, step_size=10, gamma=0.5)\n",
+ "# create 100 dummy data points and store in data loader class\n",
+ "x_train = torch.tensor(augmented_x.astype('float32'))\n",
+ "y_train = torch.tensor(augmented_y.transpose().astype('long'))\n",
+ "x_test= torch.tensor(data['x_test'].astype('float32'))\n",
+ "y_test = torch.tensor(data['y_test'].astype('long'))\n",
+ "\n",
+ "# load the data into a class that creates the batches\n",
+ "data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
+ "\n",
+ "# Initialize model weights\n",
+ "model.apply(weights_init)\n",
+ "\n",
+ "# loop over the dataset n_epoch times\n",
+ "n_epoch = 50\n",
+ "# store the loss and the % correct at each epoch\n",
+ "errors_train_aug = np.zeros((n_epoch))\n",
+ "errors_test_aug = np.zeros((n_epoch))\n",
+ "\n",
+ "for epoch in range(n_epoch):\n",
+ " # loop over batches\n",
+ " for i, batch in enumerate(data_loader):\n",
+ " # retrieve inputs and labels for this batch\n",
+ " x_batch, y_batch = batch\n",
+ " # zero the parameter gradients\n",
+ " optimizer.zero_grad()\n",
+ " # forward pass -- calculate model output\n",
+ " pred = model(x_batch)\n",
+ " # compute the loss\n",
+ " loss = loss_function(pred, y_batch)\n",
+ " # backward pass\n",
+ " loss.backward()\n",
+ " # SGD update\n",
+ " optimizer.step()\n",
+ "\n",
+ " # Run whole dataset to get statistics -- normally wouldn't do this\n",
+ " pred_train = model(x_train)\n",
+ " pred_test = model(x_test)\n",
+ " _, predicted_train_class = torch.max(pred_train.data, 1)\n",
+ " _, predicted_test_class = torch.max(pred_test.data, 1)\n",
+ " errors_train_aug[epoch] = 100 - 100 * (predicted_train_class == y_train).float().sum() / len(y_train)\n",
+ " errors_test_aug[epoch]= 100 - 100 * (predicted_test_class == y_test).float().sum() / len(y_test)\n",
+ " print(f'Epoch {epoch:5d}, train error {errors_train_aug[epoch]:3.2f}, test error {errors_test_aug[epoch]:3.2f}')"
+ ],
+ "metadata": {
+ "id": "hZUNrXpS_kRs"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Plot the results\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(errors_train,'r-',label='train')\n",
+ "ax.plot(errors_test,'b-',label='test')\n",
+ "ax.plot(errors_test_aug,'g-',label='test (augmented)')\n",
+ "ax.set_ylim(0,100); ax.set_xlim(0,n_epoch)\n",
+ "ax.set_xlabel('Epoch'); ax.set_ylabel('Error')\n",
+ "ax.set_title('TrainError %3.2f, Test Error %3.2f'%(errors_train_aug[-1],errors_test_aug[-1]))\n",
+ "ax.legend()\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "IcnAW4ixBnuc"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Hopefully, you should see an improvement in performance when we augment the data."
+ ],
+ "metadata": {
+ "id": "jgsR7ScJHc9b"
+ }
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap10/10_1_1D_Convolution.ipynb b/Notebooks/Chap10/10_1_1D_Convolution.ipynb
new file mode 100644
index 0000000..173fda0
--- /dev/null
+++ b/Notebooks/Chap10/10_1_1D_Convolution.ipynb
@@ -0,0 +1,387 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyML7rfAGE4gvmNUEiK5x3PS",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 10.1: 1D Convolution**\n",
+ "\n",
+ "This notebook investigates 1D convolutional layers.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
+ ],
+ "metadata": {
+ "id": "el8l05WQEO46"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "NOTE!!\n",
+ "\n",
+ "If you have the first edition of the printed book, it mistakenly refers to a convolutional filter with no spaces between the elements (i.e. a normal filter without dilation) as having dilation zero. Actually, the convention is (weirdly) that this has dilation one. And when there is one space between the elements, this is dilation two. This notebook reflects the correct convention and so will be out of sync with the printed book. If this is confusing, check the [errata](https://github.com/udlbook/udlbook/blob/main/UDL_Errata.pdf) document."
+ ],
+ "metadata": {
+ "id": "ggQrHkFZcUiV"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ],
+ "metadata": {
+ "id": "nw7k5yCtOzoK"
+ },
+ "execution_count": 1,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define a signal that we can apply convolution to\n",
+ "x = [5.2, 5.3, 5.4, 5.1, 10.1, 10.3, 9.9, 10.3, 3.2, 3.4, 3.3, 3.1]"
+ ],
+ "metadata": {
+ "id": "lSSHuoEqO3Ly"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Draw the signal\n",
+ "fig,ax = plt.subplots()\n",
+ "ax.plot(x, 'k-')\n",
+ "ax.set_xlim(0,11)\n",
+ "ax.set_ylim(0, 12)\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "zVssv_wiREc2"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now let's define a zero-padded convolution operation\n",
+ "# with a convolution kernel size of 3, a stride of 1, and a dilation of 1\n",
+ "# as in figure 10.2a-c. Write it yourself, don't call a library routine!\n",
+ "# Don't forget that Python arrays are indexed from zero, not from 1 as in the book figures\n",
+ "def conv_3_1_1_zp(x_in, omega):\n",
+ " x_out = np.zeros_like(x_in)\n",
+ " # TODO -- write this function\n",
+ " # replace this line\n",
+ " x_out = x_out\n",
+ "\n",
+ "\n",
+ "\n",
+ " return x_out"
+ ],
+ "metadata": {
+ "id": "MmfXED12RvNq"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's see what kind of things convolution can do\n",
+ "First, it can average nearby values, smoothing the function:"
+ ],
+ "metadata": {
+ "id": "Fof_Rs98Zovq"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "\n",
+ "omega = [0.33,0.33,0.33]\n",
+ "h = conv_3_1_1_zp(x, omega)\n",
+ "\n",
+ "# Check that you have computed this correctly\n",
+ "print(f\"Sum of output is {np.sum(h):3.3}, should be 71.1\")\n",
+ "\n",
+ "# Draw the signal\n",
+ "fig,ax = plt.subplots()\n",
+ "ax.plot(x, 'k-',label='before')\n",
+ "ax.plot(h, 'r-',label='after')\n",
+ "ax.set_xlim(0,11)\n",
+ "ax.set_ylim(0, 12)\n",
+ "ax.legend()\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "HOcPZR6iWXsa"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Notice how the red function is a smoothed version of the black one as it has averaged adjacent values. The first and last outputs are considerably lower than the original curve though. Make sure that you understand why!"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 10.2: Convolution for MNIST-1D**\n",
+ "\n",
+ "This notebook investigates a 1D convolutional network for MNIST-1D as in figure 10.7 and 10.8a.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
+ "!git clone https://github.com/greydanus/mnist1d"
+ ],
+ "metadata": {
+ "id": "D5yLObtZCi9J"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import os\n",
+ "import torch, torch.nn as nn\n",
+ "from torch.utils.data import TensorDataset, DataLoader\n",
+ "from torch.optim.lr_scheduler import StepLR\n",
+ "import matplotlib.pyplot as plt\n",
+ "import mnist1d\n",
+ "import random"
+ ],
+ "metadata": {
+ "id": "YrXWAH7sUWvU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "args = mnist1d.data.get_dataset_args()\n",
+ "data = mnist1d.data.get_dataset(args, path='./mnist1d_data.pkl', download=False, regenerate=False)\n",
+ "\n",
+ "# The training and test input and outputs are in\n",
+ "# data['x'], data['y'], data['x_test'], and data['y_test']\n",
+ "print(\"Examples in training set: {}\".format(len(data['y'])))\n",
+ "print(\"Examples in test set: {}\".format(len(data['y_test'])))\n",
+ "print(\"Length of each example: {}\".format(data['x'].shape[-1]))"
+ ],
+ "metadata": {
+ "id": "twI72ZCrCt5z"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Load in the data\n",
+ "train_data_x = data['x'].transpose()\n",
+ "train_data_y = data['y']\n",
+ "val_data_x = data['x_test'].transpose()\n",
+ "val_data_y = data['y_test']\n",
+ "# Print out sizes\n",
+ "print(\"Train data: %d examples (columns), each of which has %d dimensions (rows)\"%((train_data_x.shape[1],train_data_x.shape[0])))\n",
+ "print(\"Validation data: %d examples (columns), each of which has %d dimensions (rows)\"%((val_data_x.shape[1],val_data_x.shape[0])))"
+ ],
+ "metadata": {
+ "id": "8bKADvLHbiV5"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define the network"
+ ],
+ "metadata": {
+ "id": "_sFvRDGrl4qe"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# There are 40 input dimensions and 10 output dimensions for this data\n",
+ "# The inputs correspond to the 40 offsets in the MNIST1D template.\n",
+ "D_i = 40\n",
+ "# The outputs correspond to the 10 digits\n",
+ "D_o = 10\n",
+ "\n",
+ "\n",
+ "# TODO Create a model with the following layers\n",
+ "# 1. Convolutional layer, (input=length 40 and 1 channel, kernel size 3, stride 2, padding=\"valid\", 15 output channels )\n",
+ "# 2. ReLU\n",
+ "# 3. Convolutional layer, (input=length 19 and 15 channels, kernel size 3, stride 2, padding=\"valid\", 15 output channels )\n",
+ "# 4. ReLU\n",
+ "# 5. Convolutional layer, (input=length 9 and 15 channels, kernel size 3, stride 2, padding=\"valid\", 15 output channels)\n",
+ "# 6. ReLU\n",
+ "# 7. Flatten (converts 4x15) to length 60\n",
+ "# 8. Linear layer (input size = 60, output size = 10)\n",
+ "# References:\n",
+ "# https://pytorch.org/docs/1.13/generated/torch.nn.Conv1d.html?highlight=conv1d#torch.nn.Conv1d\n",
+ "# https://pytorch.org/docs/stable/generated/torch.nn.Flatten.html\n",
+ "# https://pytorch.org/docs/1.13/generated/torch.nn.Linear.html?highlight=linear#torch.nn.Linear\n",
+ "\n",
+ "# NOTE THAT THE CONVOLUTIONAL LAYERS NEED TO TAKE THE NUMBER OF INPUT CHANNELS AS A PARAMETER\n",
+ "# AND NOT THE INPUT SIZE.\n",
+ "\n",
+ "# Replace the following function:\n",
+ "model = nn.Sequential(\n",
+ "nn.Flatten(),\n",
+ "nn.Linear(40, 100),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(100, 100),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(100, 10))\n",
+ "\n",
+ "\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "FslroPJJffrh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# He initialization of weights\n",
+ "def weights_init(layer_in):\n",
+ " if isinstance(layer_in, nn.Linear):\n",
+ " nn.init.kaiming_uniform_(layer_in.weight)\n",
+ " layer_in.bias.data.fill_(0.0)"
+ ],
+ "metadata": {
+ "id": "YgLaex1pfhqz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# choose cross entropy loss function (equation 5.24 in the loss notes)\n",
+ "loss_function = nn.CrossEntropyLoss()\n",
+ "# construct SGD optimizer and initialize learning rate and momentum\n",
+ "optimizer = torch.optim.SGD(model.parameters(), lr = 0.05, momentum=0.9)\n",
+ "# object that decreases learning rate by half every 20 epochs\n",
+ "scheduler = StepLR(optimizer, step_size=20, gamma=0.5)\n",
+ "# create 100 dummy data points and store in data loader class\n",
+ "x_train = torch.tensor(train_data_x.transpose().astype('float32'))\n",
+ "y_train = torch.tensor(train_data_y.astype('long')).long()\n",
+ "x_val= torch.tensor(val_data_x.transpose().astype('float32'))\n",
+ "y_val = torch.tensor(val_data_y.astype('long')).long()\n",
+ "\n",
+ "# load the data into a class that creates the batches\n",
+ "data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
+ "\n",
+ "# Initialize model weights\n",
+ "model.apply(weights_init)\n",
+ "\n",
+ "# loop over the dataset n_epoch times\n",
+ "n_epoch = 100\n",
+ "# store the loss and the % correct at each epoch\n",
+ "losses_train = np.zeros((n_epoch))\n",
+ "errors_train = np.zeros((n_epoch))\n",
+ "losses_val = np.zeros((n_epoch))\n",
+ "errors_val = np.zeros((n_epoch))\n",
+ "\n",
+ "for epoch in range(n_epoch):\n",
+ " # loop over batches\n",
+ " for i, data in enumerate(data_loader):\n",
+ " # retrieve inputs and labels for this batch\n",
+ " x_batch, y_batch = data\n",
+ " # zero the parameter gradients\n",
+ " optimizer.zero_grad()\n",
+ " # forward pass -- calculate model output\n",
+ " pred = model(x_batch[:,None,:])\n",
+ " # compute the loss\n",
+ " loss = loss_function(pred, y_batch)\n",
+ " # backward pass\n",
+ " loss.backward()\n",
+ " # SGD update\n",
+ " optimizer.step()\n",
+ "\n",
+ " # Run whole dataset to get statistics -- normally wouldn't do this\n",
+ " pred_train = model(x_train[:,None,:])\n",
+ " pred_val = model(x_val[:,None,:])\n",
+ " _, predicted_train_class = torch.max(pred_train.data, 1)\n",
+ " _, predicted_val_class = torch.max(pred_val.data, 1)\n",
+ " errors_train[epoch] = 100 - 100 * (predicted_train_class == y_train).float().sum() / len(y_train)\n",
+ " errors_val[epoch]= 100 - 100 * (predicted_val_class == y_val).float().sum() / len(y_val)\n",
+ " losses_train[epoch] = loss_function(pred_train, y_train).item()\n",
+ " losses_val[epoch]= loss_function(pred_val, y_val).item()\n",
+ " print(f'Epoch {epoch:5d}, train loss {losses_train[epoch]:.6f}, train error {errors_train[epoch]:3.2f}, val loss {losses_val[epoch]:.6f}, percent error {errors_val[epoch]:3.2f}')\n",
+ "\n",
+ " # tell scheduler to consider updating learning rate\n",
+ " scheduler.step()\n",
+ "\n",
+ "# Plot the results\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(errors_train,'r-',label='train')\n",
+ "ax.plot(errors_val,'b-',label='validation')\n",
+ "ax.set_ylim(0,100); ax.set_xlim(0,n_epoch)\n",
+ "ax.set_xlabel('Epoch'); ax.set_ylabel('Error')\n",
+ "ax.set_title('Part I: Validation Result %3.2f'%(errors_val[-1]))\n",
+ "ax.legend()\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "NYw8I_3mmX5c"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap10/10_3_2D_Convolution.ipynb b/Notebooks/Chap10/10_3_2D_Convolution.ipynb
new file mode 100644
index 0000000..c304cc0
--- /dev/null
+++ b/Notebooks/Chap10/10_3_2D_Convolution.ipynb
@@ -0,0 +1,436 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyNDaU2KKZDyY9Ea7vm/fNxo",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 10.3: 2D Convolution**\n",
+ "\n",
+ "This notebook investigates the 2D convolution operation. It asks you to hand code the convolution so we can be sure that we are computing the same thing as in PyTorch. The next notebook uses the convolutional layers in PyTorch directly.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "VB_crnDGASX-"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import torch\n",
+ "# Set to print in reasonable form\n",
+ "np.set_printoptions(precision=3, floatmode=\"fixed\")\n",
+ "torch.set_printoptions(precision=3)"
+ ],
+ "metadata": {
+ "id": "YAoWDUb_DezG"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "This routine performs convolution in PyTorch"
+ ],
+ "metadata": {
+ "id": "eAwYWXzAElHG"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Perform convolution in PyTorch\n",
+ "def conv_pytorch(image, conv_weights, stride=1, pad =1):\n",
+ " # Convert image and kernel to tensors\n",
+ " image_tensor = torch.from_numpy(image) # (batchSize, channelsIn, imageHeightIn, =imageWidthIn)\n",
+ " conv_weights_tensor = torch.from_numpy(conv_weights) # (channelsOut, channelsIn, kernelHeight, kernelWidth)\n",
+ " # Do the convolution\n",
+ " output_tensor = torch.nn.functional.conv2d(image_tensor, conv_weights_tensor, stride=stride, padding=pad)\n",
+ " # Convert back from PyTorch and return\n",
+ " return(output_tensor.numpy()) # (batchSize channelsOut imageHeightOut imageHeightIn)"
+ ],
+ "metadata": {
+ "id": "xsmUIN-3BlWr"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "First we'll start with the simplest 2D convolution. Just one channel in and one channel out. A single image in the batch."
+ ],
+ "metadata": {
+ "id": "A3Sm8bUWtDNO"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Perform convolution in numpy\n",
+ "def conv_numpy_1(image, weights, pad=1):\n",
+ "\n",
+ " # Perform zero padding\n",
+ " if pad != 0:\n",
+ " image = np.pad(image, ((0, 0), (0 ,0), (pad, pad), (pad, pad)),'constant')\n",
+ "\n",
+ " # Get sizes of image array and kernel weights\n",
+ " batchSize, channelsIn, imageHeightIn, imageWidthIn = image.shape\n",
+ " channelsOut, channelsIn, kernelHeight, kernelWidth = weights.shape\n",
+ "\n",
+ " # Get size of output arrays\n",
+ " imageHeightOut = np.floor(1 + imageHeightIn - kernelHeight).astype(int)\n",
+ " imageWidthOut = np.floor(1 + imageWidthIn - kernelWidth).astype(int)\n",
+ "\n",
+ " # Create output\n",
+ " out = np.zeros((batchSize, channelsOut, imageHeightOut, imageWidthOut), dtype=np.float32)\n",
+ "\n",
+ " # !!!!!! NOTE THERE IS A SUBTLETY HERE !!!!!!!!\n",
+ " # I have padded the image with zeros above, so it is surrouned by a \"ring\" of zeros\n",
+ " # That means that the image indexes are all off by one\n",
+ " # This actually makes your code simpler\n",
+ "\n",
+ " for c_y in range(imageHeightOut):\n",
+ " for c_x in range(imageWidthOut):\n",
+ " for c_kernel_y in range(kernelHeight):\n",
+ " for c_kernel_x in range(kernelWidth):\n",
+ " # TODO -- Retrieve the image pixel and the weight from the convolution\n",
+ " # Only one image in batch, one input channel and one output channel, so these indices should all be zero\n",
+ " # Replace the two lines below\n",
+ " this_pixel_value = 1.0\n",
+ " this_weight = 1.0\n",
+ "\n",
+ "\n",
+ " # Multiply these together and add to the output at this position\n",
+ " out[0, 0, c_y, c_x] += np.sum(this_pixel_value * this_weight)\n",
+ "\n",
+ " return out"
+ ],
+ "metadata": {
+ "id": "EF8FWONVLo1Q"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set random seed so we always get same answer\n",
+ "np.random.seed(1)\n",
+ "n_batch = 1\n",
+ "image_height = 4\n",
+ "image_width = 6\n",
+ "channels_in = 1\n",
+ "kernel_size = 3\n",
+ "channels_out = 1\n",
+ "\n",
+ "# Create random input image\n",
+ "input_image= np.random.normal(size=(n_batch, channels_in, image_height, image_width))\n",
+ "# Create random convolution kernel weights\n",
+ "conv_weights = np.random.normal(size=(channels_out, channels_in, kernel_size, kernel_size))\n",
+ "\n",
+ "# Perform convolution using PyTorch\n",
+ "conv_results_pytorch = conv_pytorch(input_image, conv_weights, stride=1, pad=1)\n",
+ "print(\"PyTorch Results\")\n",
+ "print(conv_results_pytorch)\n",
+ "\n",
+ "# Perform convolution in numpy\n",
+ "print(\"Your results\")\n",
+ "conv_results_numpy = conv_numpy_1(input_image, conv_weights)\n",
+ "print(conv_results_numpy)"
+ ],
+ "metadata": {
+ "id": "iw9KqXZTHN8v"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's now add in the possibility of using different strides"
+ ],
+ "metadata": {
+ "id": "IYj_lxeGzaHX"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Perform convolution in numpy\n",
+ "def conv_numpy_2(image, weights, stride=1, pad=1):\n",
+ "\n",
+ " # Perform zero padding\n",
+ " if pad != 0:\n",
+ " image = np.pad(image, ((0, 0), (0 ,0), (pad, pad), (pad, pad)),'constant')\n",
+ "\n",
+ " # Get sizes of image array and kernel weights\n",
+ " batchSize, channelsIn, imageHeightIn, imageWidthIn = image.shape\n",
+ " channelsOut, channelsIn, kernelHeight, kernelWidth = weights.shape\n",
+ "\n",
+ " # Get size of output arrays\n",
+ " imageHeightOut = np.floor(1 + (imageHeightIn - kernelHeight) / stride).astype(int)\n",
+ " imageWidthOut = np.floor(1 + (imageWidthIn - kernelWidth) / stride).astype(int)\n",
+ "\n",
+ " # Create output\n",
+ " out = np.zeros((batchSize, channelsOut, imageHeightOut, imageWidthOut), dtype=np.float32)\n",
+ "\n",
+ " for c_y in range(imageHeightOut):\n",
+ " for c_x in range(imageWidthOut):\n",
+ " for c_kernel_y in range(kernelHeight):\n",
+ " for c_kernel_x in range(kernelWidth):\n",
+ " # TODO -- Retrieve the image pixel and the weight from the convolution\n",
+ " # Only one image in batch, one input channel and one output channel, so these indices should all be zero\n",
+ " # Replace the two lines below\n",
+ " this_pixel_value = 1.0\n",
+ " this_weight = 1.0\n",
+ "\n",
+ "\n",
+ " # Multiply these together and add to the output at this position\n",
+ " out[0, 0, c_y, c_x] += np.sum(this_pixel_value * this_weight)\n",
+ "\n",
+ " return out"
+ ],
+ "metadata": {
+ "id": "GiujmLhqHN1F"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set random seed so we always get same answer\n",
+ "np.random.seed(1)\n",
+ "n_batch = 1\n",
+ "image_height = 12\n",
+ "image_width = 10\n",
+ "channels_in = 1\n",
+ "kernel_size = 3\n",
+ "channels_out = 1\n",
+ "stride = 2\n",
+ "\n",
+ "# Create random input image\n",
+ "input_image= np.random.normal(size=(n_batch, channels_in, image_height, image_width))\n",
+ "# Create random convolution kernel weights\n",
+ "conv_weights = np.random.normal(size=(channels_out, channels_in, kernel_size, kernel_size))\n",
+ "\n",
+ "# Perform convolution using PyTorch\n",
+ "conv_results_pytorch = conv_pytorch(input_image, conv_weights, stride, pad=1)\n",
+ "print(\"PyTorch Results\")\n",
+ "print(conv_results_pytorch)\n",
+ "\n",
+ "# Perform convolution in numpy\n",
+ "print(\"Your results\")\n",
+ "conv_results_numpy = conv_numpy_2(input_image, conv_weights, stride, pad=1)\n",
+ "print(conv_results_numpy)"
+ ],
+ "metadata": {
+ "id": "FeJy6Bvozgxq"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now we'll introduce multiple input and output channels"
+ ],
+ "metadata": {
+ "id": "3flq1Wan2gX-"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Perform convolution in numpy\n",
+ "def conv_numpy_3(image, weights, stride=1, pad=1):\n",
+ "\n",
+ " # Perform zero padding\n",
+ " if pad != 0:\n",
+ " image = np.pad(image, ((0, 0), (0 ,0), (pad, pad), (pad, pad)),'constant')\n",
+ "\n",
+ " # Get sizes of image array and kernel weights\n",
+ " batchSize, channelsIn, imageHeightIn, imageWidthIn = image.shape\n",
+ " channelsOut, channelsIn, kernelHeight, kernelWidth = weights.shape\n",
+ "\n",
+ " # Get size of output arrays\n",
+ " imageHeightOut = np.floor(1 + (imageHeightIn - kernelHeight) / stride).astype(int)\n",
+ " imageWidthOut = np.floor(1 + (imageWidthIn - kernelWidth) / stride).astype(int)\n",
+ "\n",
+ " # Create output\n",
+ " out = np.zeros((batchSize, channelsOut, imageHeightOut, imageWidthOut), dtype=np.float32)\n",
+ "\n",
+ " for c_y in range(imageHeightOut):\n",
+ " for c_x in range(imageWidthOut):\n",
+ " for c_channel_out in range(channelsOut):\n",
+ " for c_channel_in in range(channelsIn):\n",
+ " for c_kernel_y in range(kernelHeight):\n",
+ " for c_kernel_x in range(kernelWidth):\n",
+ " # TODO -- Retrieve the image pixel and the weight from the convolution\n",
+ " # Only one image in batch so this index should be zero\n",
+ " # Replace the two lines below\n",
+ " this_pixel_value = 1.0\n",
+ " this_weight = 1.0\n",
+ "\n",
+ " # Multiply these together and add to the output at this position\n",
+ " out[0, c_channel_out, c_y, c_x] += np.sum(this_pixel_value * this_weight)\n",
+ " return out"
+ ],
+ "metadata": {
+ "id": "AvdRWGiU2ppX"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set random seed so we always get same answer\n",
+ "np.random.seed(1)\n",
+ "n_batch = 1\n",
+ "image_height = 4\n",
+ "image_width = 6\n",
+ "channels_in = 5\n",
+ "kernel_size = 3\n",
+ "channels_out = 2\n",
+ "\n",
+ "# Create random input image\n",
+ "input_image= np.random.normal(size=(n_batch, channels_in, image_height, image_width))\n",
+ "# Create random convolution kernel weights\n",
+ "conv_weights = np.random.normal(size=(channels_out, channels_in, kernel_size, kernel_size))\n",
+ "\n",
+ "# Perform convolution using PyTorch\n",
+ "conv_results_pytorch = conv_pytorch(input_image, conv_weights, stride=1, pad=1)\n",
+ "print(\"PyTorch Results\")\n",
+ "print(conv_results_pytorch)\n",
+ "\n",
+ "# Perform convolution in numpy\n",
+ "print(\"Your results\")\n",
+ "conv_results_numpy = conv_numpy_3(input_image, conv_weights, stride=1, pad=1)\n",
+ "print(conv_results_numpy)"
+ ],
+ "metadata": {
+ "id": "mdSmjfvY4li2"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now we'll do the full convolution with multiple images (batch size > 1), and multiple input channels, multiple output channels."
+ ],
+ "metadata": {
+ "id": "Q2MUFebdsJbH"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Perform convolution in numpy\n",
+ "def conv_numpy_4(image, weights, stride=1, pad=1):\n",
+ "\n",
+ " # Perform zero padding\n",
+ " if pad != 0:\n",
+ " image = np.pad(image, ((0, 0), (0 ,0), (pad, pad), (pad, pad)),'constant')\n",
+ "\n",
+ " # Get sizes of image array and kernel weights\n",
+ " batchSize, channelsIn, imageHeightIn, imageWidthIn = image.shape\n",
+ " channelsOut, channelsIn, kernelHeight, kernelWidth = weights.shape\n",
+ "\n",
+ " # Get size of output arrays\n",
+ " imageHeightOut = np.floor(1 + (imageHeightIn - kernelHeight) / stride).astype(int)\n",
+ " imageWidthOut = np.floor(1 + (imageWidthIn - kernelWidth) / stride).astype(int)\n",
+ "\n",
+ " # Create output\n",
+ " out = np.zeros((batchSize, channelsOut, imageHeightOut, imageWidthOut), dtype=np.float32)\n",
+ "\n",
+ " for c_batch in range(batchSize):\n",
+ " for c_y in range(imageHeightOut):\n",
+ " for c_x in range(imageWidthOut):\n",
+ " for c_channel_out in range(channelsOut):\n",
+ " for c_channel_in in range(channelsIn):\n",
+ " for c_kernel_y in range(kernelHeight):\n",
+ " for c_kernel_x in range(kernelWidth):\n",
+ " # TODO -- Retrieve the image pixel and the weight from the convolution\n",
+ " # Replace the two lines below\n",
+ " this_pixel_value = 1.0\n",
+ " this_weight = 1.0\n",
+ "\n",
+ "\n",
+ "\n",
+ " # Multiply these together and add to the output at this position\n",
+ " out[c_batch, c_channel_out, c_y, c_x] += np.sum(this_pixel_value * this_weight)\n",
+ " return out"
+ ],
+ "metadata": {
+ "id": "5WePF-Y-sC1y"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "1w2GEBtqAM2P"
+ },
+ "outputs": [],
+ "source": [
+ "# Set random seed so we always get same answer\n",
+ "np.random.seed(1)\n",
+ "n_batch = 2\n",
+ "image_height = 4\n",
+ "image_width = 6\n",
+ "channels_in = 5\n",
+ "kernel_size = 3\n",
+ "channels_out = 2\n",
+ "\n",
+ "# Create random input image\n",
+ "input_image= np.random.normal(size=(n_batch, channels_in, image_height, image_width))\n",
+ "# Create random convolution kernel weights\n",
+ "conv_weights = np.random.normal(size=(channels_out, channels_in, kernel_size, kernel_size))\n",
+ "\n",
+ "# Perform convolution using PyTorch\n",
+ "conv_results_pytorch = conv_pytorch(input_image, conv_weights, stride=1, pad=1)\n",
+ "print(\"PyTorch Results\")\n",
+ "print(conv_results_pytorch)\n",
+ "\n",
+ "# Perform convolution in numpy\n",
+ "print(\"Your results\")\n",
+ "conv_results_numpy = conv_numpy_4(input_image, conv_weights, stride=1, pad=1)\n",
+ "print(conv_results_numpy)"
+ ]
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap10/10_4_Downsampling_and_Upsampling.ipynb b/Notebooks/Chap10/10_4_Downsampling_and_Upsampling.ipynb
new file mode 100644
index 0000000..1bb2c08
--- /dev/null
+++ b/Notebooks/Chap10/10_4_Downsampling_and_Upsampling.ipynb
@@ -0,0 +1,520 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyMbSR8fzpXvO6TIQdO7bI0H",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 10.4: Downsampling and Upsampling**\n",
+ "\n",
+ "This notebook investigates the down sampling and downsampling methods discussed in section 10.4 of the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from PIL import Image\n",
+ "from numpy import asarray"
+ ],
+ "metadata": {
+ "id": "YrXWAH7sUWvU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define 4 by 4 original patch\n",
+ "orig_4_4 = np.array([[1, 3, 5,3 ], [6,2,0,8], [4,6,1,4], [2,8,0,3]])\n",
+ "print(orig_4_4)"
+ ],
+ "metadata": {
+ "id": "WPRoJcC_JXE2"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def subsample(x_in):\n",
+ " x_out = np.zeros(( int(np.ceil(x_in.shape[0]/2)), int(np.ceil(x_in.shape[1]/2)) ))\n",
+ " # TO DO -- write the subsampling routine\n",
+ " # Replace this line\n",
+ " x_out = x_out\n",
+ "\n",
+ "\n",
+ " return x_out"
+ ],
+ "metadata": {
+ "id": "qneyOiZRJubi"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "print(\"Original:\")\n",
+ "print(orig_4_4)\n",
+ "print(\"Subsampled:\")\n",
+ "print(subsample(orig_4_4))"
+ ],
+ "metadata": {
+ "id": "O_i0y72_JwGZ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's try that on an image to get a feel for how it works:"
+ ],
+ "metadata": {
+ "id": "AobyC8IILbCO"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap10/test_image.png"
+ ],
+ "metadata": {
+ "id": "3dJEo-6DM-Py"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# load the image\n",
+ "image = Image.open('test_image.png')\n",
+ "# convert image to numpy array\n",
+ "data = asarray(image)\n",
+ "data_subsample = subsample(data);\n",
+ "\n",
+ "plt.figure(figsize=(5,5))\n",
+ "plt.imshow(data, cmap='gray')\n",
+ "plt.show()\n",
+ "\n",
+ "plt.figure(figsize=(5,5))\n",
+ "plt.imshow(data_subsample, cmap='gray')\n",
+ "plt.show()\n",
+ "\n",
+ "data_subsample2 = subsample(data_subsample)\n",
+ "plt.figure(figsize=(5,5))\n",
+ "plt.imshow(data_subsample2, cmap='gray')\n",
+ "plt.show()\n",
+ "\n",
+ "data_subsample3 = subsample(data_subsample2)\n",
+ "plt.figure(figsize=(5,5))\n",
+ "plt.imshow(data_subsample3, cmap='gray')\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "HCZVutk6NB6B"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now let's try max-pooling\n",
+ "def maxpool(x_in):\n",
+ " x_out = np.zeros(( int(np.floor(x_in.shape[0]/2)), int(np.floor(x_in.shape[1]/2)) ))\n",
+ " # TO DO -- write the maxpool routine\n",
+ " # Replace this line\n",
+ " x_out = x_out\n",
+ "\n",
+ " return x_out"
+ ],
+ "metadata": {
+ "id": "Z99uYehaPtJa"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "print(\"Original:\")\n",
+ "print(orig_4_4)\n",
+ "print(\"Maxpooled:\")\n",
+ "print(maxpool(orig_4_4))"
+ ],
+ "metadata": {
+ "id": "J4KMTMmG9P44"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's see what Rick looks like:\n",
+ "data_maxpool = maxpool(data);\n",
+ "\n",
+ "plt.figure(figsize=(5,5))\n",
+ "plt.imshow(data, cmap='gray')\n",
+ "plt.show()\n",
+ "\n",
+ "plt.figure(figsize=(5,5))\n",
+ "plt.imshow(data_maxpool, cmap='gray')\n",
+ "plt.show()\n",
+ "\n",
+ "data_maxpool2 = maxpool(data_maxpool)\n",
+ "plt.figure(figsize=(5,5))\n",
+ "plt.imshow(data_maxpool2, cmap='gray')\n",
+ "plt.show()\n",
+ "\n",
+ "data_maxpool3 = maxpool(data_maxpool2)\n",
+ "plt.figure(figsize=(5,5))\n",
+ "plt.imshow(data_maxpool3, cmap='gray')\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "0ES0sB8t9Wyv"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "You can see that the stripes on his shirt gradually turn to white because we keep retaining the brightest local pixels."
+ ],
+ "metadata": {
+ "id": "nMtSdBGlAktq"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Finally, let's try mean pooling\n",
+ "def meanpool(x_in):\n",
+ " x_out = np.zeros(( int(np.floor(x_in.shape[0]/2)), int(np.floor(x_in.shape[1]/2)) ))\n",
+ " # TO DO -- write the meanpool routine\n",
+ " # Replace this line\n",
+ " x_out = x_out\n",
+ "\n",
+ " return x_out"
+ ],
+ "metadata": {
+ "id": "ZQBjBtmB_aGQ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "print(\"Original:\")\n",
+ "print(orig_4_4)\n",
+ "print(\"Meanpooled:\")\n",
+ "print(meanpool(orig_4_4))"
+ ],
+ "metadata": {
+ "id": "N4VDlWNt_8dp"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's see what Rick looks like:\n",
+ "data_meanpool = meanpool(data);\n",
+ "\n",
+ "plt.figure(figsize=(5,5))\n",
+ "plt.imshow(data, cmap='gray')\n",
+ "plt.show()\n",
+ "\n",
+ "plt.figure(figsize=(5,5))\n",
+ "plt.imshow(data_meanpool, cmap='gray')\n",
+ "plt.show()\n",
+ "\n",
+ "data_meanpool2 = meanpool(data_maxpool)\n",
+ "plt.figure(figsize=(5,5))\n",
+ "plt.imshow(data_meanpool2, cmap='gray')\n",
+ "plt.show()\n",
+ "\n",
+ "data_meanpool3 = meanpool(data_meanpool2)\n",
+ "plt.figure(figsize=(5,5))\n",
+ "plt.imshow(data_meanpool3, cmap='gray')\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "Lkg5zUYo_-IV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Notice that the three low resolution images look quite different. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 10.5: Convolution for MNIST**\n",
+ "\n",
+ "This notebook builds a proper network for 2D convolution. It works with the MNIST dataset (figure 15.15a), which was the original classic dataset for classifying images. The network will take a 28x28 grayscale image and classify it into one of 10 classes representing a digit.\n",
+ "\n",
+ "The code is adapted from https://nextjournal.com/gkoehler/pytorch-mnist\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import torch\n",
+ "import torchvision\n",
+ "import torch.nn as nn\n",
+ "import torch.nn.functional as F\n",
+ "import torch.optim as optim\n",
+ "import matplotlib.pyplot as plt\n",
+ "import random"
+ ],
+ "metadata": {
+ "id": "YrXWAH7sUWvU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run this once to load the train and test data straight into a dataloader class\n",
+ "# that will provide the batches\n",
+ "batch_size_train = 64\n",
+ "batch_size_test = 1000\n",
+ "train_loader = torch.utils.data.DataLoader(\n",
+ " torchvision.datasets.MNIST('/files/', train=True, download=True,\n",
+ " transform=torchvision.transforms.Compose([\n",
+ " torchvision.transforms.ToTensor(),\n",
+ " torchvision.transforms.Normalize(\n",
+ " (0.1307,), (0.3081,))\n",
+ " ])),\n",
+ " batch_size=batch_size_train, shuffle=True)\n",
+ "\n",
+ "test_loader = torch.utils.data.DataLoader(\n",
+ " torchvision.datasets.MNIST('/files/', train=False, download=True,\n",
+ " transform=torchvision.transforms.Compose([\n",
+ " torchvision.transforms.ToTensor(),\n",
+ " torchvision.transforms.Normalize(\n",
+ " (0.1307,), (0.3081,))\n",
+ " ])),\n",
+ " batch_size=batch_size_test, shuffle=True)"
+ ],
+ "metadata": {
+ "id": "wScBGXXFVadm"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's draw some of the training data\n",
+ "examples = enumerate(test_loader)\n",
+ "batch_idx, (example_data, example_targets) = next(examples)\n",
+ "\n",
+ "fig = plt.figure()\n",
+ "for i in range(6):\n",
+ " plt.subplot(2,3,i+1)\n",
+ " plt.tight_layout()\n",
+ " plt.imshow(example_data[i][0], cmap='gray', interpolation='none')\n",
+ " plt.title(\"Ground Truth: {}\".format(example_targets[i]))\n",
+ " plt.xticks([])\n",
+ " plt.yticks([])\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "8bKADvLHbiV5"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define the network. This is a more typical way to define a network than the sequential structure. We define a class for the network, and define the parameters in the constructor. Then we use a function called forward to actually run the network. It's easy to see how you might use residual connections in this format."
+ ],
+ "metadata": {
+ "id": "_sFvRDGrl4qe"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "from os import X_OK\n",
+ "# TODO Change this class to implement\n",
+ "# 1. A valid convolution with kernel size 5, 1 input channel and 10 output channels\n",
+ "# 2. A max pooling operation over a 2x2 area\n",
+ "# 3. A Relu\n",
+ "# 4. A valid convolution with kernel size 5, 10 input channels and 20 output channels\n",
+ "# 5. A 2D Dropout layer\n",
+ "# 6. A max pooling operation over a 2x2 area\n",
+ "# 7. A relu\n",
+ "# 8. A flattening operation\n",
+ "# 9. A fully connected layer mapping from (whatever dimensions we are at-- find out using .shape) to 50\n",
+ "# 10. A ReLU\n",
+ "# 11. A fully connected layer mapping from 50 to 10 dimensions\n",
+ "# 12. A softmax function.\n",
+ "\n",
+ "# Replace this class which implements a minimal network (which still does okay)\n",
+ "class Net(nn.Module):\n",
+ " def __init__(self):\n",
+ " super(Net, self).__init__()\n",
+ " # Valid convolution, 1 channel in, 2 channels out, stride 1, kernel size = 3\n",
+ " self.conv1 = nn.Conv2d(1, 2, kernel_size=3)\n",
+ " # Dropout for convolutions\n",
+ " self.drop = nn.Dropout2d()\n",
+ " # Fully connected layer\n",
+ " self.fc1 = nn.Linear(338, 10)\n",
+ "\n",
+ " def forward(self, x):\n",
+ " x = self.conv1(x)\n",
+ " x = self.drop(x)\n",
+ " x = F.max_pool2d(x,2)\n",
+ " x = F.relu(x)\n",
+ " x = x.flatten(1)\n",
+ " x = self.fc1(x)\n",
+ " x = F.log_softmax(x)\n",
+ " return x\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "EQkvw2KOPVl7"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# He initialization of weights\n",
+ "def weights_init(layer_in):\n",
+ " if isinstance(layer_in, nn.Linear):\n",
+ " nn.init.kaiming_uniform_(layer_in.weight)\n",
+ " layer_in.bias.data.fill_(0.0)"
+ ],
+ "metadata": {
+ "id": "qWZtkCZcU_dg"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Create network\n",
+ "model = Net()\n",
+ "# Initialize model weights\n",
+ "model.apply(weights_init)\n",
+ "# Define optimizer\n",
+ "optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.5)"
+ ],
+ "metadata": {
+ "id": "FslroPJJffrh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Main training routine\n",
+ "def train(epoch):\n",
+ " model.train()\n",
+ " # Get each\n",
+ " for batch_idx, (data, target) in enumerate(train_loader):\n",
+ " optimizer.zero_grad()\n",
+ " output = model(data)\n",
+ " loss = F.nll_loss(output, target)\n",
+ " loss.backward()\n",
+ " optimizer.step()\n",
+ " # Store results\n",
+ " if batch_idx % 10 == 0:\n",
+ " print('Train Epoch: {} [{}/{}]\\tLoss: {:.6f}'.format(\n",
+ " epoch, batch_idx * len(data), len(train_loader.dataset), loss.item()))"
+ ],
+ "metadata": {
+ "id": "xKQd9PzkQ766"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run on test data\n",
+ "def test():\n",
+ " model.eval()\n",
+ " test_loss = 0\n",
+ " correct = 0\n",
+ " with torch.no_grad():\n",
+ " for data, target in test_loader:\n",
+ " output = model(data)\n",
+ " test_loss += F.nll_loss(output, target, size_average=False).item()\n",
+ " pred = output.data.max(1, keepdim=True)[1]\n",
+ " correct += pred.eq(target.data.view_as(pred)).sum()\n",
+ " test_loss /= len(test_loader.dataset)\n",
+ " print('\\nTest set: Avg. loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\\n'.format(\n",
+ " test_loss, correct, len(test_loader.dataset),\n",
+ " 100. * correct / len(test_loader.dataset)))"
+ ],
+ "metadata": {
+ "id": "Byn-f7qWRLxX"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Get initial performance\n",
+ "test()\n",
+ "# Train for three epochs\n",
+ "n_epochs = 3\n",
+ "for epoch in range(1, n_epochs + 1):\n",
+ " train(epoch)\n",
+ " test()"
+ ],
+ "metadata": {
+ "id": "YgLaex1pfhqz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run network on data we got before and show predictions\n",
+ "output = model(example_data)\n",
+ "\n",
+ "fig = plt.figure()\n",
+ "for i in range(10):\n",
+ " plt.subplot(5,5,i+1)\n",
+ " plt.tight_layout()\n",
+ " plt.imshow(example_data[i][0], cmap='gray', interpolation='none')\n",
+ " plt.title(\"Prediction: {}\".format(\n",
+ " output.data.max(1, keepdim=True)[1][i].item()))\n",
+ " plt.xticks([])\n",
+ " plt.yticks([])\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "o7fRUAy9Se1B"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap10/test_image.png b/Notebooks/Chap10/test_image.png
new file mode 100644
index 0000000..426b6f7
Binary files /dev/null and b/Notebooks/Chap10/test_image.png differ
diff --git a/Notebooks/Chap11/11_1_Shattered_Gradients.ipynb b/Notebooks/Chap11/11_1_Shattered_Gradients.ipynb
new file mode 100644
index 0000000..6ef827d
--- /dev/null
+++ b/Notebooks/Chap11/11_1_Shattered_Gradients.ipynb
@@ -0,0 +1,392 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyMLKg5ZmXqojcVrZD5BGm9g",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 11.1: Shattered gradients**\n",
+ "\n",
+ "This notebook investigates the phenomenon of shattered gradients as discussed in section 11.1.1. It replicates some of the experiments in [Balduzzi et al. (2017)](https://arxiv.org/abs/1702.08591).\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "pOZ6Djz0dhoy"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ],
+ "metadata": {
+ "id": "iaFyNGhU21VJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "First let's define a neural network. We'll initialize both the weights and biases randomly with Glorot initialization (He initialization without the factor of two)"
+ ],
+ "metadata": {
+ "id": "YcNlAxnE3XXn"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# K is width, D is number of hidden units in each layer\n",
+ "def init_params(K, D):\n",
+ " # Set seed so we always get the same random numbers\n",
+ " np.random.seed(1)\n",
+ "\n",
+ " # Input layer\n",
+ " D_i = 1\n",
+ " # Output layer\n",
+ " D_o = 1\n",
+ "\n",
+ " # Glorot initialization\n",
+ " sigma_sq_omega = 1.0/D\n",
+ "\n",
+ " # Make empty lists\n",
+ " all_weights = [None] * (K+1)\n",
+ " all_biases = [None] * (K+1)\n",
+ "\n",
+ " # Create parameters for input and output layers\n",
+ " all_weights[0] = np.random.normal(size=(D, D_i))*np.sqrt(sigma_sq_omega)\n",
+ " all_weights[-1] = np.random.normal(size=(D_o, D)) * np.sqrt(sigma_sq_omega)\n",
+ " all_biases[0] = np.random.normal(size=(D,1))* np.sqrt(sigma_sq_omega)\n",
+ " all_biases[-1]= np.random.normal(size=(D_o,1))* np.sqrt(sigma_sq_omega)\n",
+ "\n",
+ " # Create intermediate layers\n",
+ " for layer in range(1,K):\n",
+ " all_weights[layer] = np.random.normal(size=(D,D))*np.sqrt(sigma_sq_omega)\n",
+ " all_biases[layer] = np.random.normal(size=(D,1))* np.sqrt(sigma_sq_omega)\n",
+ "\n",
+ " return all_weights, all_biases"
+ ],
+ "metadata": {
+ "id": "kr-q7hc23Bn9"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The next two functions define the forward pass of the algorithm"
+ ],
+ "metadata": {
+ "id": "kwcn5z7-dq_1"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the Rectified Linear Unit (ReLU) function\n",
+ "def ReLU(preactivation):\n",
+ " activation = preactivation.clip(0.0)\n",
+ " return activation\n",
+ "\n",
+ "def forward_pass(net_input, all_weights, all_biases):\n",
+ "\n",
+ " # Retrieve number of layers\n",
+ " K = len(all_weights) -1\n",
+ "\n",
+ " # We'll store the pre-activations at each layer in a list \"all_f\"\n",
+ " # and the activations in a second list[all_h].\n",
+ " all_f = [None] * (K+1)\n",
+ " all_h = [None] * (K+1)\n",
+ "\n",
+ " #For convenience, we'll set\n",
+ " # all_h[0] to be the input, and all_f[K] will be the output\n",
+ " all_h[0] = net_input\n",
+ "\n",
+ " # Run through the layers, calculating all_f[0...K-1] and all_h[1...K]\n",
+ " for layer in range(K):\n",
+ " # Update preactivations and activations at this layer according to eqn 7.5\n",
+ " all_f[layer] = all_biases[layer] + np.matmul(all_weights[layer], all_h[layer])\n",
+ " all_h[layer+1] = ReLU(all_f[layer])\n",
+ "\n",
+ " # Compute the output from the last hidden layer\n",
+ " all_f[K] = all_biases[K] + np.matmul(all_weights[K], all_h[K])\n",
+ "\n",
+ " # Retrieve the output\n",
+ " net_output = all_f[K]\n",
+ "\n",
+ " return net_output, all_f, all_h"
+ ],
+ "metadata": {
+ "id": "_2w-Tr7G3sYq"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The next two functions compute the gradient of the output with respect to the input using the back propagation algorithm."
+ ],
+ "metadata": {
+ "id": "aM2l7QafeC8T"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# We'll need the indicator function\n",
+ "def indicator_function(x):\n",
+ " x_in = np.array(x)\n",
+ " x_in[x_in>=0] = 1\n",
+ " x_in[x_in<0] = 0\n",
+ " return x_in\n",
+ "\n",
+ "# Main backward pass routine\n",
+ "def calc_input_output_gradient(x_in, all_weights, all_biases):\n",
+ "\n",
+ " # Run the forward pass\n",
+ " y, all_f, all_h = forward_pass(x_in, all_weights, all_biases)\n",
+ "\n",
+ " # We'll store the derivatives dl_dweights and dl_dbiases in lists as well\n",
+ " all_dl_dweights = [None] * (K+1)\n",
+ " all_dl_dbiases = [None] * (K+1)\n",
+ " # And we'll store the derivatives of the loss with respect to the activation and preactivations in lists\n",
+ " all_dl_df = [None] * (K+1)\n",
+ " all_dl_dh = [None] * (K+1)\n",
+ " # Again for convenience we'll stick with the convention that all_h[0] is the net input and all_f[k] in the net output\n",
+ "\n",
+ " # Compute derivatives of net output with respect to loss\n",
+ " all_dl_df[K] = np.ones_like(all_f[K])\n",
+ "\n",
+ " # Now work backwards through the network\n",
+ " for layer in range(K,-1,-1):\n",
+ " all_dl_dbiases[layer] = np.array(all_dl_df[layer])\n",
+ " all_dl_dweights[layer] = np.matmul(all_dl_df[layer], all_h[layer].transpose())\n",
+ "\n",
+ " all_dl_dh[layer] = np.matmul(all_weights[layer].transpose(), all_dl_df[layer])\n",
+ "\n",
+ " if layer > 0:\n",
+ " all_dl_df[layer-1] = indicator_function(all_f[layer-1]) * all_dl_dh[layer]\n",
+ "\n",
+ "\n",
+ " return all_dl_dh[0],y"
+ ],
+ "metadata": {
+ "id": "DwR3eGMgV8bl"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Double check we have the gradient correct using finite differences"
+ ],
+ "metadata": {
+ "id": "Ar_VmraReSWe"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "D = 200; K = 3\n",
+ "# Initialize parameters\n",
+ "all_weights, all_biases = init_params(K,D)\n",
+ "\n",
+ "x = np.ones((1,1))\n",
+ "dydx,y = calc_input_output_gradient(x, all_weights, all_biases)\n",
+ "\n",
+ "# Offset for finite gradients\n",
+ "delta = 0.00000001\n",
+ "x1 = x\n",
+ "y1,*_ = forward_pass(x1, all_weights, all_biases)\n",
+ "x2 = x+delta\n",
+ "y2,*_ = forward_pass(x2, all_weights, all_biases)\n",
+ "# Finite difference calculation\n",
+ "dydx_fd = (y2-y1)/delta\n",
+ "\n",
+ "print(\"Gradient calculation=%f, Finite difference gradient=%f\"%(dydx.squeeze(),dydx_fd.squeeze()))\n"
+ ],
+ "metadata": {
+ "id": "KJpQPVd36Haq"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Helper function that computes the derivatives for a 1D array of input values and plots them."
+ ],
+ "metadata": {
+ "id": "YC-LAYRKtbxp"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def plot_derivatives(K, D):\n",
+ "\n",
+ " # Initialize parameters\n",
+ " all_weights, all_biases = init_params(K,D)\n",
+ "\n",
+ " x_in = np.arange(-2,2, 4.0/256.0)\n",
+ " x_in = np.resize(x_in, (1,len(x_in)))\n",
+ " dydx,y = calc_input_output_gradient(x_in, all_weights, all_biases)\n",
+ "\n",
+ " fig,ax = plt.subplots()\n",
+ " ax.plot(np.squeeze(x_in), np.squeeze(dydx), 'b-')\n",
+ " ax.set_xlim(-2,2)\n",
+ " ax.set_xlabel('Input, $x$')\n",
+ " ax.set_ylabel('Gradient, $dy/dx$')\n",
+ " ax.set_title('No layers = %d'%(K))\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "uJr5eDe648jF"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Build a model with one hidden layer and 200 neurons and plot derivatives\n",
+ "D = 200; K = 1\n",
+ "plot_derivatives(K,D)\n",
+ "\n",
+ "# TODO -- Interpret this result\n",
+ "# Why does the plot have some flat regions?\n",
+ "\n",
+ "# TODO -- Add code to plot the derivatives for models with 24 and 50 hidden layers\n",
+ "# with 200 neurons per layer\n",
+ "\n",
+ "# TODO -- Why does this graph not have visible flat regions?\n",
+ "\n",
+ "# TODO -- Why does the magnitude of the gradients decrease as we increase the number\n",
+ "# of hidden layers\n",
+ "\n",
+ "# TODO -- Do you find this a convincing replication of the experiment in the original paper? (I don't)\n",
+ "# Can you help me find why I have failed to replicate this result? udlbookmail@gmail.com"
+ ],
+ "metadata": {
+ "id": "56gTMTCb49KO"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's look at the autocorrelation function now"
+ ],
+ "metadata": {
+ "id": "f_0zjQbxuROQ"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def autocorr(dydx):\n",
+ " # TODO -- compute the autocorrelation function\n",
+ " # Use the numpy function \"correlate\" with the mode set to \"same\"\n",
+ " # Replace this line:\n",
+ " ac = np.ones((256,1))\n",
+ "\n",
+ " return ac"
+ ],
+ "metadata": {
+ "id": "ggnO8hfoRN1e"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Helper function to plot the autocorrelation function and normalize so correlation is one with offset of zero"
+ ],
+ "metadata": {
+ "id": "EctWSV1RuddK"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def plot_autocorr(K, D):\n",
+ "\n",
+ " # Initialize parameters\n",
+ " all_weights, all_biases = init_params(K,D)\n",
+ "\n",
+ " x_in = np.arange(-2.0,2.0, 4.0/256)\n",
+ " x_in = np.resize(x_in, (1,len(x_in)))\n",
+ " dydx,y = calc_input_output_gradient(x_in, all_weights, all_biases)\n",
+ " ac = autocorr(np.squeeze(dydx))\n",
+ " ac = ac / ac[128]\n",
+ "\n",
+ " y = ac[128:]\n",
+ " x = np.squeeze(x_in)[128:]\n",
+ " fig,ax = plt.subplots()\n",
+ " ax.plot(x,y, 'b-')\n",
+ " ax.set_xlim([0,2])\n",
+ " ax.set_xlabel('Distance')\n",
+ " ax.set_ylabel('Autocorrelation')\n",
+ " ax.set_title('No layers = %d'%(K))\n",
+ " plt.show()\n"
+ ],
+ "metadata": {
+ "id": "2LKlZ9u_WQXN"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Plot the autocorrelation functions\n",
+ "D = 200; K =1\n",
+ "plot_autocorr(K,D)\n",
+ "D = 200; K =50\n",
+ "plot_autocorr(K,D)\n",
+ "\n",
+ "# TODO -- Do you find this a convincing replication of the experiment in the original paper? (I don't)\n",
+ "# Can you help me find why I have failed to replicate this result?"
+ ],
+ "metadata": {
+ "id": "RD9JTdjNWw6p"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap11/11_2_Residual_Networks.ipynb b/Notebooks/Chap11/11_2_Residual_Networks.ipynb
new file mode 100644
index 0000000..613920f
--- /dev/null
+++ b/Notebooks/Chap11/11_2_Residual_Networks.ipynb
@@ -0,0 +1,277 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyMXS3SPB4cS/4qxix0lH/Hq",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 11.2: Residual Networks**\n",
+ "\n",
+ "This notebook adapts the networks for MNIST1D to use residual connections.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
+ "!git clone https://github.com/greydanus/mnist1d"
+ ],
+ "metadata": {
+ "id": "D5yLObtZCi9J"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import os\n",
+ "import torch, torch.nn as nn\n",
+ "from torch.utils.data import TensorDataset, DataLoader\n",
+ "from torch.optim.lr_scheduler import StepLR\n",
+ "import matplotlib.pyplot as plt\n",
+ "import mnist1d\n",
+ "import random"
+ ],
+ "metadata": {
+ "id": "YrXWAH7sUWvU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "args = mnist1d.data.get_dataset_args()\n",
+ "data = mnist1d.data.get_dataset(args, path='./mnist1d_data.pkl', download=False, regenerate=False)\n",
+ "\n",
+ "# The training and test input and outputs are in\n",
+ "# data['x'], data['y'], data['x_test'], and data['y_test']\n",
+ "print(\"Examples in training set: {}\".format(len(data['y'])))\n",
+ "print(\"Examples in test set: {}\".format(len(data['y_test'])))\n",
+ "print(\"Length of each example: {}\".format(data['x'].shape[-1]))"
+ ],
+ "metadata": {
+ "id": "twI72ZCrCt5z"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Load in the data\n",
+ "train_data_x = data['x'].transpose()\n",
+ "train_data_y = data['y']\n",
+ "val_data_x = data['x_test'].transpose()\n",
+ "val_data_y = data['y_test']\n",
+ "# Print out sizes\n",
+ "print(\"Train data: %d examples (columns), each of which has %d dimensions (rows)\"%((train_data_x.shape[1],train_data_x.shape[0])))\n",
+ "print(\"Validation data: %d examples (columns), each of which has %d dimensions (rows)\"%((val_data_x.shape[1],val_data_x.shape[0])))"
+ ],
+ "metadata": {
+ "id": "8bKADvLHbiV5"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define the network"
+ ],
+ "metadata": {
+ "id": "_sFvRDGrl4qe"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# There are 40 input dimensions and 10 output dimensions for this data\n",
+ "# The inputs correspond to the 40 offsets in the MNIST1D template.\n",
+ "D_i = 40\n",
+ "# The outputs correspond to the 10 digits\n",
+ "D_o = 10\n",
+ "\n",
+ "\n",
+ "# We will adapt this model to have residual connections around the linear layers\n",
+ "# This is the same model we used in practical 8.1, but we can't use the sequential\n",
+ "# class for residual networks (which aren't strictly sequential). Hence, I've rewritten\n",
+ "# it as a model that inherits from a base class\n",
+ "\n",
+ "class ResidualNetwork(torch.nn.Module):\n",
+ " def __init__(self, input_size, output_size, hidden_size=100):\n",
+ " super(ResidualNetwork, self).__init__()\n",
+ " self.linear1 = nn.Linear(input_size, hidden_size)\n",
+ " self.linear2 = nn.Linear(hidden_size, hidden_size)\n",
+ " self.linear3 = nn.Linear(hidden_size, hidden_size)\n",
+ " self.linear4 = nn.Linear(hidden_size, output_size)\n",
+ " print(\"Initialized MLPBase model with {} parameters\".format(self.count_params()))\n",
+ "\n",
+ " def count_params(self):\n",
+ " return sum([p.view(-1).shape[0] for p in self.parameters()])\n",
+ "\n",
+ "# TODO -- Add residual connections to this model\n",
+ "# The order of operations within each block should similar to figure 11.5b\n",
+ "# ie., linear1 first, ReLU+linear2 in first residual block, ReLU+linear3 in second residual block), linear4 at end\n",
+ "# Replace this function\n",
+ " def forward(self, x):\n",
+ " h1 = self.linear1(x).relu()\n",
+ " h2 = self.linear2(h1).relu()\n",
+ " h3 = self.linear3(h2).relu()\n",
+ " return self.linear4(h3)\n"
+ ],
+ "metadata": {
+ "id": "FslroPJJffrh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# He initialization of weights\n",
+ "def weights_init(layer_in):\n",
+ " if isinstance(layer_in, nn.Linear):\n",
+ " nn.init.kaiming_uniform_(layer_in.weight)\n",
+ " layer_in.bias.data.fill_(0.0)"
+ ],
+ "metadata": {
+ "id": "YgLaex1pfhqz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "#Define the model\n",
+ "model = ResidualNetwork(40, 10)\n",
+ "\n",
+ "# choose cross entropy loss function (equation 5.24 in the loss notes)\n",
+ "loss_function = nn.CrossEntropyLoss()\n",
+ "# construct SGD optimizer and initialize learning rate and momentum\n",
+ "optimizer = torch.optim.SGD(model.parameters(), lr = 0.05, momentum=0.9)\n",
+ "# object that decreases learning rate by half every 20 epochs\n",
+ "scheduler = StepLR(optimizer, step_size=20, gamma=0.5)\n",
+ "# convert data to torch tensors\n",
+ "x_train = torch.tensor(train_data_x.transpose().astype('float32'))\n",
+ "y_train = torch.tensor(train_data_y.astype('long'))\n",
+ "x_val= torch.tensor(val_data_x.transpose().astype('float32'))\n",
+ "y_val = torch.tensor(val_data_y.astype('long'))\n",
+ "\n",
+ "# load the data into a class that creates the batches\n",
+ "data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
+ "\n",
+ "# Initialize model weights\n",
+ "model.apply(weights_init)\n",
+ "\n",
+ "# loop over the dataset n_epoch times\n",
+ "n_epoch = 100\n",
+ "# store the loss and the % correct at each epoch\n",
+ "losses_train = np.zeros((n_epoch))\n",
+ "errors_train = np.zeros((n_epoch))\n",
+ "losses_val = np.zeros((n_epoch))\n",
+ "errors_val = np.zeros((n_epoch))\n",
+ "\n",
+ "for epoch in range(n_epoch):\n",
+ " # loop over batches\n",
+ " for i, data in enumerate(data_loader):\n",
+ " # retrieve inputs and labels for this batch\n",
+ " x_batch, y_batch = data\n",
+ " # zero the parameter gradients\n",
+ " optimizer.zero_grad()\n",
+ " # forward pass -- calculate model output\n",
+ " pred = model(x_batch)\n",
+ " # compute the loss\n",
+ " loss = loss_function(pred, y_batch)\n",
+ " # backward pass\n",
+ " loss.backward()\n",
+ " # SGD update\n",
+ " optimizer.step()\n",
+ "\n",
+ " # Run whole dataset to get statistics -- normally wouldn't do this\n",
+ " pred_train = model(x_train)\n",
+ " pred_val = model(x_val)\n",
+ " _, predicted_train_class = torch.max(pred_train.data, 1)\n",
+ " _, predicted_val_class = torch.max(pred_val.data, 1)\n",
+ " errors_train[epoch] = 100 - 100 * (predicted_train_class == y_train).float().sum() / len(y_train)\n",
+ " errors_val[epoch]= 100 - 100 * (predicted_val_class == y_val).float().sum() / len(y_val)\n",
+ " losses_train[epoch] = loss_function(pred_train, y_train).item()\n",
+ " losses_val[epoch]= loss_function(pred_val, y_val).item()\n",
+ " print(f'Epoch {epoch:5d}, train loss {losses_train[epoch]:.6f}, train error {errors_train[epoch]:3.2f}, val loss {losses_val[epoch]:.6f}, percent error {errors_val[epoch]:3.2f}')\n",
+ "\n",
+ " # tell scheduler to consider updating learning rate\n",
+ " scheduler.step()"
+ ],
+ "metadata": {
+ "id": "NYw8I_3mmX5c"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Plot the results\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(errors_train,'r-',label='train')\n",
+ "ax.plot(errors_val,'b-',label='test')\n",
+ "ax.set_ylim(0,100); ax.set_xlim(0,n_epoch)\n",
+ "ax.set_xlabel('Epoch'); ax.set_ylabel('Error')\n",
+ "ax.set_title('TrainError %3.2f, Val Error %3.2f'%(errors_train[-1],errors_val[-1]))\n",
+ "ax.legend()\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "CcP_VyEmE2sv"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The primary motivation of residual networks is to allow training of much deeper networks. \n",
+ "\n",
+ "TODO: Try running this network with and without the residual connections. Does adding the residual connections change the performance?"
+ ],
+ "metadata": {
+ "id": "wMmqhmxuAx0M"
+ }
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap11/11_3_Batch_Normalization.ipynb b/Notebooks/Chap11/11_3_Batch_Normalization.ipynb
new file mode 100644
index 0000000..8efd2f5
--- /dev/null
+++ b/Notebooks/Chap11/11_3_Batch_Normalization.ipynb
@@ -0,0 +1,330 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyPVeAd3eDpEOCFh8CVyr1zz",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 11.3: Batch normalization**\n",
+ "\n",
+ "This notebook investigates the use of batch normalization in residual networks.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
+ "!git clone https://github.com/greydanus/mnist1d"
+ ],
+ "metadata": {
+ "id": "D5yLObtZCi9J"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import os\n",
+ "import torch, torch.nn as nn\n",
+ "from torch.utils.data import TensorDataset, DataLoader\n",
+ "from torch.optim.lr_scheduler import StepLR\n",
+ "import matplotlib.pyplot as plt\n",
+ "import mnist1d\n",
+ "import random"
+ ],
+ "metadata": {
+ "id": "YrXWAH7sUWvU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "args = mnist1d.data.get_dataset_args()\n",
+ "data = mnist1d.data.get_dataset(args, path='./mnist1d_data.pkl', download=False, regenerate=False)\n",
+ "\n",
+ "# The training and test input and outputs are in\n",
+ "# data['x'], data['y'], data['x_test'], and data['y_test']\n",
+ "print(\"Examples in training set: {}\".format(len(data['y'])))\n",
+ "print(\"Examples in test set: {}\".format(len(data['y_test'])))\n",
+ "print(\"Length of each example: {}\".format(data['x'].shape[-1]))"
+ ],
+ "metadata": {
+ "id": "twI72ZCrCt5z"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Load in the data\n",
+ "train_data_x = data['x'].transpose()\n",
+ "train_data_y = data['y']\n",
+ "val_data_x = data['x_test'].transpose()\n",
+ "val_data_y = data['y_test']\n",
+ "# Print out sizes\n",
+ "print(\"Train data: %d examples (columns), each of which has %d dimensions (rows)\"%((train_data_x.shape[1],train_data_x.shape[0])))\n",
+ "print(\"Validation data: %d examples (columns), each of which has %d dimensions (rows)\"%((val_data_x.shape[1],val_data_x.shape[0])))"
+ ],
+ "metadata": {
+ "id": "8bKADvLHbiV5"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def print_variance(name, data):\n",
+ " # First dimension(rows) is batch elements\n",
+ " # Second dimension(columns) is neurons.\n",
+ " np_data = data.detach().numpy()\n",
+ " # Compute variance across neurons and average these variances over members of the batch\n",
+ " neuron_variance = np.mean(np.var(np_data, axis=0))\n",
+ " # Print out the name and the variance\n",
+ " print(\"%s variance=%f\"%(name,neuron_variance))"
+ ],
+ "metadata": {
+ "id": "3bBpJIV-N-lt"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# He initialization of weights\n",
+ "def weights_init(layer_in):\n",
+ " if isinstance(layer_in, nn.Linear):\n",
+ " nn.init.kaiming_uniform_(layer_in.weight)\n",
+ " layer_in.bias.data.fill_(0.0)"
+ ],
+ "metadata": {
+ "id": "YgLaex1pfhqz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def run_one_step_of_model(model, x_train, y_train):\n",
+ " # choose cross entropy loss function (equation 5.24 in the loss notes)\n",
+ " loss_function = nn.CrossEntropyLoss()\n",
+ " # construct SGD optimizer and initialize learning rate and momentum\n",
+ " optimizer = torch.optim.SGD(model.parameters(), lr = 0.05, momentum=0.9)\n",
+ "\n",
+ " # load the data into a class that creates the batches\n",
+ " data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=200, shuffle=True, worker_init_fn=np.random.seed(1))\n",
+ "\n",
+ " # Initialize model weights\n",
+ " model.apply(weights_init)\n",
+ "\n",
+ " # Get a batch\n",
+ " for i, data in enumerate(data_loader):\n",
+ " # retrieve inputs and labels for this batch\n",
+ " x_batch, y_batch = data\n",
+ " # zero the parameter gradients\n",
+ " optimizer.zero_grad()\n",
+ " # forward pass -- calculate model output\n",
+ " pred = model(x_batch)\n",
+ " # compute the loss\n",
+ " loss = loss_function(pred, y_batch)\n",
+ " # backward pass\n",
+ " loss.backward()\n",
+ " # SGD update\n",
+ " optimizer.step()\n",
+ " # Break out of this loop -- we just want to see the first\n",
+ " # iteration, but usually we would continue\n",
+ " break"
+ ],
+ "metadata": {
+ "id": "DFlu45pORQEz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# convert training data to torch tensors\n",
+ "x_train = torch.tensor(train_data_x.transpose().astype('float32'))\n",
+ "y_train = torch.tensor(train_data_y.astype('long'))"
+ ],
+ "metadata": {
+ "id": "i7Q0ScWgRe4G"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# This is a simple residual model with 5 residual branches in a row\n",
+ "class ResidualNetwork(torch.nn.Module):\n",
+ " def __init__(self, input_size, output_size, hidden_size=100):\n",
+ " super(ResidualNetwork, self).__init__()\n",
+ " self.linear1 = nn.Linear(input_size, hidden_size)\n",
+ " self.linear2 = nn.Linear(hidden_size, hidden_size)\n",
+ " self.linear3 = nn.Linear(hidden_size, hidden_size)\n",
+ " self.linear4 = nn.Linear(hidden_size, hidden_size)\n",
+ " self.linear5 = nn.Linear(hidden_size, hidden_size)\n",
+ " self.linear6 = nn.Linear(hidden_size, hidden_size)\n",
+ " self.linear7 = nn.Linear(hidden_size, output_size)\n",
+ "\n",
+ " def count_params(self):\n",
+ " return sum([p.view(-1).shape[0] for p in self.parameters()])\n",
+ "\n",
+ " def forward(self, x):\n",
+ " print_variance(\"Input\",x)\n",
+ " f = self.linear1(x)\n",
+ " print_variance(\"First preactivation\",f)\n",
+ " res1 = f+ self.linear2(f.relu())\n",
+ " print_variance(\"After first residual connection\",res1)\n",
+ " res2 = res1 + self.linear3(res1.relu())\n",
+ " print_variance(\"After second residual connection\",res2)\n",
+ " res3 = res2 + self.linear4(res2.relu())\n",
+ " print_variance(\"After third residual connection\",res3)\n",
+ " res4 = res3 + self.linear5(res3.relu())\n",
+ " print_variance(\"After fourth residual connection\",res4)\n",
+ " res5 = res4 + self.linear6(res4.relu())\n",
+ " print_variance(\"After fifth residual connection\",res5)\n",
+ " return self.linear7(res5)"
+ ],
+ "metadata": {
+ "id": "FslroPJJffrh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the model and run for one step\n",
+ "# Monitoring the variance at each point in the network\n",
+ "n_hidden = 100\n",
+ "n_input = 40\n",
+ "n_output = 10\n",
+ "model = ResidualNetwork(n_input, n_output, n_hidden)\n",
+ "run_one_step_of_model(model, x_train, y_train)"
+ ],
+ "metadata": {
+ "id": "NYw8I_3mmX5c"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Notice that the variance roughly doubles at each step so it increases exponentially as in figure 11.6b in the book."
+ ],
+ "metadata": {
+ "id": "0kZUlWkkW8jE"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO Adapt the residual network below to add a batch norm operation\n",
+ "# before the contents of each residual link as in figure 11.6c in the book\n",
+ "# Use the torch function nn.BatchNorm1d\n",
+ "class ResidualNetworkWithBatchNorm(torch.nn.Module):\n",
+ " def __init__(self, input_size, output_size, hidden_size=100):\n",
+ " super(ResidualNetworkWithBatchNorm, self).__init__()\n",
+ " self.linear1 = nn.Linear(input_size, hidden_size)\n",
+ " self.linear2 = nn.Linear(hidden_size, hidden_size)\n",
+ " self.linear3 = nn.Linear(hidden_size, hidden_size)\n",
+ " self.linear4 = nn.Linear(hidden_size, hidden_size)\n",
+ " self.linear5 = nn.Linear(hidden_size, hidden_size)\n",
+ " self.linear6 = nn.Linear(hidden_size, hidden_size)\n",
+ " self.linear7 = nn.Linear(hidden_size, output_size)\n",
+ "\n",
+ " def count_params(self):\n",
+ " return sum([p.view(-1).shape[0] for p in self.parameters()])\n",
+ "\n",
+ " def forward(self, x):\n",
+ " print_variance(\"Input\",x)\n",
+ " f = self.linear1(x)\n",
+ " print_variance(\"First preactivation\",f)\n",
+ " res1 = f+ self.linear2(f.relu())\n",
+ " print_variance(\"After first residual connection\",res1)\n",
+ " res2 = res1 + self.linear3(res1.relu())\n",
+ " print_variance(\"After second residual connection\",res2)\n",
+ " res3 = res2 + self.linear4(res2.relu())\n",
+ " print_variance(\"After third residual connection\",res3)\n",
+ " res4 = res3 + self.linear5(res3.relu())\n",
+ " print_variance(\"After fourth residual connection\",res4)\n",
+ " res5 = res4 + self.linear6(res4.relu())\n",
+ " print_variance(\"After fifth residual connection\",res5)\n",
+ " return self.linear7(res5)"
+ ],
+ "metadata": {
+ "id": "5JvMmaRITKGd"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the model\n",
+ "n_hidden = 100\n",
+ "n_input = 40\n",
+ "n_output = 10\n",
+ "model = ResidualNetworkWithBatchNorm(n_input, n_output, n_hidden)\n",
+ "run_one_step_of_model(model, x_train, y_train)"
+ ],
+ "metadata": {
+ "id": "2U3DnlH9Uw6c"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Note that the variance now increases linearly as in figure 11.6c."
+ ],
+ "metadata": {
+ "id": "R_ucFq9CXq8D"
+ }
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap12/12_1_Self_Attention.ipynb b/Notebooks/Chap12/12_1_Self_Attention.ipynb
new file mode 100644
index 0000000..eb584b8
--- /dev/null
+++ b/Notebooks/Chap12/12_1_Self_Attention.ipynb
@@ -0,0 +1,375 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyOKrX9gmuhl9+KwscpZKr3u",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 12.1: Self Attention**\n",
+ "\n",
+ "This notebook builds a self-attention mechanism from scratch, as discussed in section 12.2 of the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ],
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The self-attention mechanism maps $N$ inputs $\\mathbf{x}_{n}\\in\\mathbb{R}^{D}$ and returns $N$ outputs $\\mathbf{x}'_{n}\\in \\mathbb{R}^{D}$. \n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "9OJkkoNqCVK2"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set seed so we get the same random numbers\n",
+ "np.random.seed(3)\n",
+ "# Number of inputs\n",
+ "N = 3\n",
+ "# Number of dimensions of each input\n",
+ "D = 4\n",
+ "# Create an empty list\n",
+ "all_x = []\n",
+ "# Create elements x_n and append to list\n",
+ "for n in range(N):\n",
+ " all_x.append(np.random.normal(size=(D,1)))\n",
+ "# Print out the list\n",
+ "print(all_x)\n"
+ ],
+ "metadata": {
+ "id": "oAygJwLiCSri"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "We'll also need the weights and biases for the keys, queries, and values (equations 12.2 and 12.4)"
+ ],
+ "metadata": {
+ "id": "W2iHFbtKMaDp"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set seed so we get the same random numbers\n",
+ "np.random.seed(0)\n",
+ "\n",
+ "# Choose random values for the parameters\n",
+ "omega_q = np.random.normal(size=(D,D))\n",
+ "omega_k = np.random.normal(size=(D,D))\n",
+ "omega_v = np.random.normal(size=(D,D))\n",
+ "beta_q = np.random.normal(size=(D,1))\n",
+ "beta_k = np.random.normal(size=(D,1))\n",
+ "beta_v = np.random.normal(size=(D,1))"
+ ],
+ "metadata": {
+ "id": "79TSK7oLMobe"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's compute the queries, keys, and values for each input"
+ ],
+ "metadata": {
+ "id": "VxaKQtP3Ng6R"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Make three lists to store queries, keys, and values\n",
+ "all_queries = []\n",
+ "all_keys = []\n",
+ "all_values = []\n",
+ "# For every input\n",
+ "for x in all_x:\n",
+ " # TODO -- compute the keys, queries and values.\n",
+ " # Replace these three lines\n",
+ " query = np.ones_like(x)\n",
+ " key = np.ones_like(x)\n",
+ " value = np.ones_like(x)\n",
+ "\n",
+ " all_queries.append(query)\n",
+ " all_keys.append(key)\n",
+ " all_values.append(value)"
+ ],
+ "metadata": {
+ "id": "TwDK2tfdNmw9"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "We'll need a softmax function (equation 12.5) -- here, it will take a list of arbitrary numbers and return a list where the elements are non-negative and sum to one\n"
+ ],
+ "metadata": {
+ "id": "Se7DK6PGPSUk"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def softmax(items_in):\n",
+ "\n",
+ " # TODO Compute the elements of items_out\n",
+ " # Replace this line\n",
+ " items_out = items_in.copy()\n",
+ "\n",
+ " return items_out ;"
+ ],
+ "metadata": {
+ "id": "u93LIcE5PoiM"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now compute the self attention values:"
+ ],
+ "metadata": {
+ "id": "8aJVhbKDW7lm"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Create emptymlist for output\n",
+ "all_x_prime = []\n",
+ "\n",
+ "# For each output\n",
+ "for n in range(N):\n",
+ " # Create list for dot products of query N with all keys\n",
+ " all_km_qn = []\n",
+ " # Compute the dot products\n",
+ " for key in all_keys:\n",
+ " # TODO -- compute the appropriate dot product\n",
+ " # Replace this line\n",
+ " dot_product = 1\n",
+ "\n",
+ " # Store dot product\n",
+ " all_km_qn.append(dot_product)\n",
+ "\n",
+ " # Compute dot product\n",
+ " attention = softmax(all_km_qn)\n",
+ " # Print result (should be positive sum to one)\n",
+ " print(\"Attentions for output \", n)\n",
+ " print(attention)\n",
+ "\n",
+ " # TODO: Compute a weighted sum of all of the values according to the attention\n",
+ " # (equation 12.3)\n",
+ " # Replace this line\n",
+ " x_prime = np.zeros((D,1))\n",
+ "\n",
+ " all_x_prime.append(x_prime)\n",
+ "\n",
+ "\n",
+ "# Print out true values to check you have it correct\n",
+ "print(\"x_prime_0_calculated:\", all_x_prime[0].transpose())\n",
+ "print(\"x_prime_0_true: [[ 0.94744244 -0.24348429 -0.91310441 -0.44522983]]\")\n",
+ "print(\"x_prime_1_calculated:\", all_x_prime[1].transpose())\n",
+ "print(\"x_prime_1_true: [[ 1.64201168 -0.08470004 4.02764044 2.18690791]]\")\n",
+ "print(\"x_prime_2_calculated:\", all_x_prime[2].transpose())\n",
+ "print(\"x_prime_2_true: [[ 1.61949281 -0.06641533 3.96863308 2.15858316]]\")\n"
+ ],
+ "metadata": {
+ "id": "yimz-5nCW6vQ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's compute the same thing, but using matrix calculations. We'll store the $N$ inputs $\\mathbf{x}_{n}\\in\\mathbb{R}^{D}$ in the columns of a $D\\times N$ matrix, using equations 12.6 and 12.7/8.\n",
+ "\n",
+ "Note: The book uses column vectors (for compatibility with the rest of the text), but in the wider literature it is more normal to store the inputs in the rows of a matrix; in this case, the computation is the same, but all the matrices are transposed and the operations proceed in the reverse order."
+ ],
+ "metadata": {
+ "id": "PJ2vCQ_7C38K"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define softmax operation that works independently on each column\n",
+ "def softmax_cols(data_in):\n",
+ " # Exponentiate all of the values\n",
+ " exp_values = np.exp(data_in) ;\n",
+ " # Sum over columns\n",
+ " denom = np.sum(exp_values, axis = 0);\n",
+ " # Replicate denominator to N rows\n",
+ " denom = np.matmul(np.ones((data_in.shape[0],1)), denom[np.newaxis,:])\n",
+ " # Compute softmax\n",
+ " softmax = exp_values / denom\n",
+ " # return the answer\n",
+ " return softmax"
+ ],
+ "metadata": {
+ "id": "obaQBdUAMXXv"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ " # Now let's compute self attention in matrix form\n",
+ "def self_attention(X,omega_v, omega_q, omega_k, beta_v, beta_q, beta_k):\n",
+ "\n",
+ " # TODO -- Write this function\n",
+ " # 1. Compute queries, keys, and values\n",
+ " # 2. Compute dot products\n",
+ " # 3. Apply softmax to calculate attentions\n",
+ " # 4. Weight values by attentions\n",
+ " # Replace this line\n",
+ " X_prime = np.zeros_like(X);\n",
+ "\n",
+ "\n",
+ " return X_prime"
+ ],
+ "metadata": {
+ "id": "gb2WvQ3SiH8r"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Copy data into matrix\n",
+ "X = np.zeros((D, N))\n",
+ "X[:,0] = np.squeeze(all_x[0])\n",
+ "X[:,1] = np.squeeze(all_x[1])\n",
+ "X[:,2] = np.squeeze(all_x[2])\n",
+ "\n",
+ "# Run the self attention mechanism\n",
+ "X_prime = self_attention(X,omega_v, omega_q, omega_k, beta_v, beta_q, beta_k)\n",
+ "\n",
+ "# Print out the results\n",
+ "print(X_prime)"
+ ],
+ "metadata": {
+ "id": "MUOJbgJskUpl"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "If you did this correctly, the values should be the same as above.\n",
+ "\n",
+ "TODO: \n",
+ "\n",
+ "Print out the attention matrix\n",
+ "You will see that the values are quite extreme (one is very close to one and the others are very close to zero. Now we'll fix this problem by using scaled dot-product attention."
+ ],
+ "metadata": {
+ "id": "as_lRKQFpvz0"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now let's compute self attention in matrix form\n",
+ "def scaled_dot_product_self_attention(X,omega_v, omega_q, omega_k, beta_v, beta_q, beta_k):\n",
+ "\n",
+ " # TODO -- Write this function\n",
+ " # 1. Compute queries, keys, and values\n",
+ " # 2. Compute dot products\n",
+ " # 3. Scale the dot products as in equation 12.9\n",
+ " # 4. Apply softmax to calculate attentions\n",
+ " # 5. Weight values by attentions\n",
+ " # Replace this line\n",
+ " X_prime = np.zeros_like(X);\n",
+ "\n",
+ " return X_prime"
+ ],
+ "metadata": {
+ "id": "kLU7PUnnqvIh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run the self attention mechanism\n",
+ "X_prime = scaled_dot_product_self_attention(X,omega_v, omega_q, omega_k, beta_v, beta_q, beta_k)\n",
+ "\n",
+ "# Print out the results\n",
+ "print(X_prime)"
+ ],
+ "metadata": {
+ "id": "n18e3XNzmVgL"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "TODO -- Investigate whether the self-attention mechanism is covariant with respect to permutation.\n",
+ "If it is, when we permute the columns of the input matrix $\\mathbf{X}$, the columns of the output matrix $\\mathbf{X}'$ will also be permuted.\n"
+ ],
+ "metadata": {
+ "id": "QDEkIrcgrql-"
+ }
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap12/12_2_Multihead_Self_Attention.ipynb b/Notebooks/Chap12/12_2_Multihead_Self_Attention.ipynb
new file mode 100644
index 0000000..7e9b0b3
--- /dev/null
+++ b/Notebooks/Chap12/12_2_Multihead_Self_Attention.ipynb
@@ -0,0 +1,212 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyMSk8qTqDYqFnRJVZKlsue0",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 12.1: Multhead Self-Attention**\n",
+ "\n",
+ "This notebook builds a multihead self-attention mechanism as in figure 12.6\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ],
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The multihead self-attention mechanism maps $N$ inputs $\\mathbf{x}_{n}\\in\\mathbb{R}^{D}$ and returns $N$ outputs $\\mathbf{x}'_{n}\\in \\mathbb{R}^{D}$. \n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "9OJkkoNqCVK2"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set seed so we get the same random numbers\n",
+ "np.random.seed(3)\n",
+ "# Number of inputs\n",
+ "N = 6\n",
+ "# Number of dimensions of each input\n",
+ "D = 8\n",
+ "# Create an empty list\n",
+ "X = np.random.normal(size=(D,N))\n",
+ "# Print X\n",
+ "print(X)"
+ ],
+ "metadata": {
+ "id": "oAygJwLiCSri"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "We'll use two heads. We'll need the weights and biases for the keys, queries, and values (equations 12.2 and 12.4). We'll use two heads, and (as in the figure), we'll make the queries keys and values of size D/H"
+ ],
+ "metadata": {
+ "id": "W2iHFbtKMaDp"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Number of heads\n",
+ "H = 2\n",
+ "# QDV dimension\n",
+ "H_D = int(D/H)\n",
+ "\n",
+ "# Set seed so we get the same random numbers\n",
+ "np.random.seed(0)\n",
+ "\n",
+ "# Choose random values for the parameters for the first head\n",
+ "omega_q1 = np.random.normal(size=(H_D,D))\n",
+ "omega_k1 = np.random.normal(size=(H_D,D))\n",
+ "omega_v1 = np.random.normal(size=(H_D,D))\n",
+ "beta_q1 = np.random.normal(size=(H_D,1))\n",
+ "beta_k1 = np.random.normal(size=(H_D,1))\n",
+ "beta_v1 = np.random.normal(size=(H_D,1))\n",
+ "\n",
+ "# Choose random values for the parameters for the second head\n",
+ "omega_q2 = np.random.normal(size=(H_D,D))\n",
+ "omega_k2 = np.random.normal(size=(H_D,D))\n",
+ "omega_v2 = np.random.normal(size=(H_D,D))\n",
+ "beta_q2 = np.random.normal(size=(H_D,1))\n",
+ "beta_k2 = np.random.normal(size=(H_D,1))\n",
+ "beta_v2 = np.random.normal(size=(H_D,1))\n",
+ "\n",
+ "# Choose random values for the parameters\n",
+ "omega_c = np.random.normal(size=(D,D))"
+ ],
+ "metadata": {
+ "id": "79TSK7oLMobe"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's compute the multiscale self-attention"
+ ],
+ "metadata": {
+ "id": "VxaKQtP3Ng6R"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define softmax operation that works independently on each column\n",
+ "def softmax_cols(data_in):\n",
+ " # Exponentiate all of the values\n",
+ " exp_values = np.exp(data_in) ;\n",
+ " # Sum over columns\n",
+ " denom = np.sum(exp_values, axis = 0);\n",
+ " # Replicate denominator to N rows\n",
+ " denom = np.matmul(np.ones((data_in.shape[0],1)), denom[np.newaxis,:])\n",
+ " # Compute softmax\n",
+ " softmax = exp_values / denom\n",
+ " # return the answer\n",
+ " return softmax"
+ ],
+ "metadata": {
+ "id": "obaQBdUAMXXv"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ " # Now let's compute self attention in matrix form\n",
+ "def multihead_scaled_self_attention(X,omega_v1, omega_q1, omega_k1, beta_v1, beta_q1, beta_k1, omega_v2, omega_q2, omega_k2, beta_v2, beta_q2, beta_k2, omega_c):\n",
+ "\n",
+ " # TODO Write the multihead scaled self-attention mechanism.\n",
+ " # Replace this line\n",
+ " X_prime = np.zeros_like(X) ;\n",
+ "\n",
+ "\n",
+ " return X_prime"
+ ],
+ "metadata": {
+ "id": "gb2WvQ3SiH8r"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run the self attention mechanism\n",
+ "X_prime = multihead_scaled_self_attention(X,omega_v1, omega_q1, omega_k1, beta_v1, beta_q1, beta_k1, omega_v2, omega_q2, omega_k2, beta_v2, beta_q2, beta_k2, omega_c)\n",
+ "\n",
+ "# Print out the results\n",
+ "np.set_printoptions(precision=3)\n",
+ "print(\"Your answer:\")\n",
+ "print(X_prime)\n",
+ "\n",
+ "print(\"True values:\")\n",
+ "print(\"[[-21.207 -5.373 -20.933 -9.179 -11.319 -17.812]\")\n",
+ "print(\" [ -1.995 7.906 -10.516 3.452 9.863 -7.24 ]\")\n",
+ "print(\" [ 5.479 1.115 9.244 0.453 5.656 7.089]\")\n",
+ "print(\" [ -7.413 -7.416 0.363 -5.573 -6.736 -0.848]\")\n",
+ "print(\" [-11.261 -9.937 -4.848 -8.915 -13.378 -5.761]\")\n",
+ "print(\" [ 3.548 10.036 -2.244 1.604 12.113 -2.557]\")\n",
+ "print(\" [ 4.888 -5.814 2.407 3.228 -4.232 3.71 ]\")\n",
+ "print(\" [ 1.248 18.894 -6.409 3.224 19.717 -5.629]]\")\n",
+ "\n",
+ "# If your answers don't match, then make sure that you are doing the scaling, and make sure the scaling value is correct"
+ ],
+ "metadata": {
+ "id": "MUOJbgJskUpl"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap12/12_3_Tokenization.ipynb b/Notebooks/Chap12/12_3_Tokenization.ipynb
new file mode 100644
index 0000000..67c5cfc
--- /dev/null
+++ b/Notebooks/Chap12/12_3_Tokenization.ipynb
@@ -0,0 +1,341 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyP0/KodWM9Dtr2x+8MdXXH1",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 12.3: Tokenization**\n",
+ "\n",
+ "This notebook builds set of tokens from a text string as in figure 12.8 of the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "I adapted this code from *SOMEWHERE*. If anyone recognizes it, can you let me know and I will give the proper attribution or rewrite if the license is not permissive.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import re, collections"
+ ],
+ "metadata": {
+ "id": "3_WkaFO3OfLi"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "text = \"a sailor went to sea sea sea \"+\\\n",
+ " \"to see what he could see see see \"+\\\n",
+ " \"but all that he could see see see \"+\\\n",
+ " \"was the bottom of the deep blue sea sea sea\""
+ ],
+ "metadata": {
+ "id": "tVZVuauIXmJk"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Tokenize the input sentence To begin with the tokens are the individual letters and the whitespace token. So, we represent each word in terms of these tokens with spaces between the tokens to delineate them.\n",
+ "\n",
+ "The tokenized text is stored in a structure that represents each word as tokens together with the count of how often that word occurs. We'll call this the *vocabulary*."
+ ],
+ "metadata": {
+ "id": "fF2RBrouWV5w"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def initialize_vocabulary(text):\n",
+ " vocab = collections.defaultdict(int)\n",
+ " words = text.strip().split()\n",
+ " for word in words:\n",
+ " vocab[' '.join(list(word)) + ' '] += 1\n",
+ " return vocab"
+ ],
+ "metadata": {
+ "id": "OfvXkLSARk4_"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "vocab = initialize_vocabulary(text)\n",
+ "print('Vocabulary: {}'.format(vocab))\n",
+ "print('Size of vocabulary: {}'.format(len(vocab)))"
+ ],
+ "metadata": {
+ "id": "aydmNqaoOpSm"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Find all the tokens in the current vocabulary and their frequencies"
+ ],
+ "metadata": {
+ "id": "fJAiCjphWsI9"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def get_tokens_and_frequencies(vocab):\n",
+ " tokens = collections.defaultdict(int)\n",
+ " for word, freq in vocab.items():\n",
+ " word_tokens = word.split()\n",
+ " for token in word_tokens:\n",
+ " tokens[token] += freq\n",
+ " return tokens"
+ ],
+ "metadata": {
+ "id": "qYi6F_K3RYsW"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "tokens = get_tokens_and_frequencies(vocab)\n",
+ "print('Tokens: {}'.format(tokens))\n",
+ "print('Number of tokens: {}'.format(len(tokens)))"
+ ],
+ "metadata": {
+ "id": "Y4LCVGnvXIwp"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Find each pair of adjacent tokens in the vocabulary\n",
+ "and count them. We will subsequently merge the most frequently occurring pair."
+ ],
+ "metadata": {
+ "id": "_-Rh1mD_Ww3b"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def get_pairs_and_counts(vocab):\n",
+ " pairs = collections.defaultdict(int)\n",
+ " for word, freq in vocab.items():\n",
+ " symbols = word.split()\n",
+ " for i in range(len(symbols)-1):\n",
+ " pairs[symbols[i],symbols[i+1]] += freq\n",
+ " return pairs"
+ ],
+ "metadata": {
+ "id": "OqJTB3UFYubH"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "pairs = get_pairs_and_counts(vocab)\n",
+ "print('Pairs: {}'.format(pairs))\n",
+ "print('Number of distinct pairs: {}'.format(len(pairs)))\n",
+ "\n",
+ "most_frequent_pair = max(pairs, key=pairs.get)\n",
+ "print('Most frequent pair: {}'.format(most_frequent_pair))"
+ ],
+ "metadata": {
+ "id": "d-zm0JBcZSjS"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Merge the instances of the best pair in the vocabulary"
+ ],
+ "metadata": {
+ "id": "pcborzqIXQFS"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def merge_pair_in_vocabulary(pair, vocab_in):\n",
+ " vocab_out = {}\n",
+ " bigram = re.escape(' '.join(pair))\n",
+ " p = re.compile(r'(?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 12.4: Decoding strategies**\n",
+ "\n",
+ "This practical investigates neural decoding from transformer models. \n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "RnIUiieJWu6e"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "!pip install transformers"
+ ],
+ "metadata": {
+ "id": "7abjZ9pMVj3k"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "from transformers import GPT2LMHeadModel, GPT2Tokenizer, set_seed\n",
+ "import torch\n",
+ "import torch.nn.functional as F\n",
+ "import numpy as np"
+ ],
+ "metadata": {
+ "id": "sMOyD0zem2Ef"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Load model and tokenizer\n",
+ "model = GPT2LMHeadModel.from_pretrained('gpt2')\n",
+ "tokenizer = GPT2Tokenizer.from_pretrained('gpt2')"
+ ],
+ "metadata": {
+ "id": "pZgfxbzKWNSR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Decoding from GPT2\n",
+ "\n",
+ "This tutorial investigates how to use GPT2 (the forerunner of GPT3) to generate text. There are a number of ways to do this that trade-off the realism of the text against the amount of variation.\n",
+ "\n",
+ "At every stage, GPT2 takes an input string and returns a probability for each of the possible subsequent tokens. We can choose what to do with these probability. We could always *greedily choose* the most likely next token, or we could draw a *sample* randomly according to the probabilities. There are also intermediate strategies such as *top-k sampling* and *nucleus sampling*, that have some controlled randomness.\n",
+ "\n",
+ "We'll also investigate *beam search* -- the idea is that rather than greedily take the next best token at each stage, we maintain a set of hypotheses (beams)as we add each subsequent token and return the most likely overall hypothesis. This is not necessarily the same result we get from greedily choosing the next token."
+ ],
+ "metadata": {
+ "id": "TfhAGy0TXEvV"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "First, let's investigate the token themselves. The code below prints out the vocabulary size and shows 20 random tokens. "
+ ],
+ "metadata": {
+ "id": "vsmO9ptzau3_"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "np.random.seed(1)\n",
+ "print(\"Number of tokens in dictionary = %d\"%(tokenizer.vocab_size))\n",
+ "for i in range(20):\n",
+ " index = np.random.randint(tokenizer.vocab_size)\n",
+ " print(\"Token: %d \"%(index)+tokenizer.decode(torch.tensor(index), skip_special_tokens=True))\n"
+ ],
+ "metadata": {
+ "id": "dmmBNS5GY_yk"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Sampling\n",
+ "\n",
+ "Each time we run GPT2 it will take in a set of tokens, and return a probability over each of the possible next tokens. The simplest thing we could do is to just draw a sample from this probability distribution each time."
+ ],
+ "metadata": {
+ "id": "MUM3kLEjbTso"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def sample_next_token(input_tokens, model, tokenizer):\n",
+ " # Run model to get prediction over next output\n",
+ " outputs = model(input_ids = input_tokens['input_ids'], attention_mask = input_tokens['attention_mask'])\n",
+ " # Find prediction\n",
+ " prob_over_tokens = F.softmax(outputs.logits, dim=-1).detach().numpy()[0,-1]\n",
+ " # TODO Draw a random token according to the probabilities\n",
+ " # next_token should be an array with an sole integer in it (as below)\n",
+ " # Use: https://numpy.org/doc/stable/reference/random/generated/numpy.random.choice.html\n",
+ " # Replace this line\n",
+ " next_token = [5000]\n",
+ "\n",
+ "\n",
+ " # Append token to sentence\n",
+ " output_tokens = input_tokens\n",
+ " output_tokens[\"input_ids\"] = torch.cat((output_tokens['input_ids'],torch.tensor([next_token])),dim=1)\n",
+ " output_tokens['attention_mask'] = torch.cat((output_tokens['attention_mask'],torch.tensor([[1]])),dim=1)\n",
+ " output_tokens['last_token_prob'] = prob_over_tokens[next_token]\n",
+ "\n",
+ " return output_tokens"
+ ],
+ "metadata": {
+ "id": "TIyNgg0FkJKO"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Expected output:\n",
+ "# \"The best thing about Bath is that they don't even change or shrink anymore.\"\n",
+ "\n",
+ "set_seed(0)\n",
+ "input_txt = \"The best thing about Bath is\"\n",
+ "input_tokens = tokenizer(input_txt, return_tensors='pt')\n",
+ "for i in range(10):\n",
+ " input_tokens = sample_next_token(input_tokens, model, tokenizer)\n",
+ " print(tokenizer.decode(input_tokens[\"input_ids\"][0], skip_special_tokens=True))\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "BHs-IWaz9MNY"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO Modify the code below by changing the number of tokens generated and the initial sentence\n",
+ "# to get a feel for how well this works. Since I didn't reset the seed, it will give a different\n",
+ "# answer every time that you run it.\n",
+ "\n",
+ "# TODO Experiment with changing this line:\n",
+ "input_txt = \"The best thing about Bath is\"\n",
+ "input_tokens = tokenizer(input_txt, return_tensors='pt')\n",
+ "# TODO Experiment with changing this line:\n",
+ "for i in range(10):\n",
+ " input_tokens = sample_next_token(input_tokens, model, tokenizer)\n",
+ " print(tokenizer.decode(input_tokens[\"input_ids\"][0], skip_special_tokens=True))"
+ ],
+ "metadata": {
+ "id": "yN98_7WqbvIe"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Greedy token selection\n",
+ "\n",
+ "You probably (correctly) got the impression that the text from pure sampling of the probability model can be kind of random. How about if we choose most likely token at each step?\n"
+ ],
+ "metadata": {
+ "id": "7eHFLCeZcmmg"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def get_best_next_token(input_tokens, model, tokenizer):\n",
+ " # Run model to get prediction over next output\n",
+ " outputs = model(input_ids = input_tokens['input_ids'], attention_mask = input_tokens['attention_mask'])\n",
+ " # Find prediction\n",
+ " prob_over_tokens = F.softmax(outputs.logits, dim=-1).detach().numpy()[0,-1]\n",
+ "\n",
+ " # TODO -- find the token index with the maximum probability\n",
+ " # It should be returns as a list (i.e., put squared brackets around it)\n",
+ " # Use https://numpy.org/doc/stable/reference/generated/numpy.argmax.html\n",
+ " # Replace this line\n",
+ " next_token = [5000]\n",
+ "\n",
+ "\n",
+ " # Append token to sentence\n",
+ " output_tokens = input_tokens\n",
+ " output_tokens[\"input_ids\"] = torch.cat((output_tokens['input_ids'],torch.tensor([next_token])),dim=1)\n",
+ " output_tokens['attention_mask'] = torch.cat((output_tokens['attention_mask'],torch.tensor([[1]])),dim=1)\n",
+ " output_tokens['last_token_prob'] = prob_over_tokens[next_token]\n",
+ " return output_tokens"
+ ],
+ "metadata": {
+ "id": "OhRzynEjxpZF"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Expected output:\n",
+ "# The best thing about Bath is that it's a place where you can go to\n",
+ "set_seed(0)\n",
+ "input_txt = \"The best thing about Bath is\"\n",
+ "input_tokens = tokenizer(input_txt, return_tensors='pt')\n",
+ "for i in range(10):\n",
+ " input_tokens = get_best_next_token(input_tokens, model, tokenizer)\n",
+ " print(tokenizer.decode(input_tokens[\"input_ids\"][0], skip_special_tokens=True))"
+ ],
+ "metadata": {
+ "id": "gKB1Mgndj-Hm"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO Modify the code below by changing the number of tokens generated and the initial sentence\n",
+ "# to get a feel for how well this works.\n",
+ "\n",
+ "# TODO Experiment with changing this line:\n",
+ "input_txt = \"The best thing about Bath is\"\n",
+ "input_tokens = tokenizer(input_txt, return_tensors='pt')\n",
+ "# TODO Experiment with changing this line:\n",
+ "for i in range(10):\n",
+ " input_tokens = get_best_next_token(input_tokens, model, tokenizer)\n",
+ " print(tokenizer.decode(input_tokens[\"input_ids\"][0], skip_special_tokens=True))"
+ ],
+ "metadata": {
+ "id": "L1YHKaYFfC0M"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Top-K sampling\n",
+ "\n",
+ "You probably noticed that the greedy strategy produces quite realistic text, but it's kind of boring. It produces generic answers. Also, if this was a chatbot, then we wouldn't necessarily want it to produce the same answer to a question each time. \n",
+ "\n",
+ "Top-K sampling is a compromise strategy that samples randomly from the top K most probable tokens. We could just choose them with a uniform distribution, or (as here) we could sample them according to their original probabilities."
+ ],
+ "metadata": {
+ "id": "1ORFXYX_gBDT"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def get_top_k_token(input_tokens, model, tokenizer, k=20):\n",
+ " # Run model to get prediction over next output\n",
+ " outputs = model(input_ids = input_tokens['input_ids'], attention_mask = input_tokens['attention_mask'])\n",
+ " # Find prediction\n",
+ " prob_over_tokens = F.softmax(outputs.logits, dim=-1).detach().numpy()[0,-1]\n",
+ "\n",
+ " # Draw a sample from the top K most likely tokens.\n",
+ " # Take copy of the probabilities and sort from largest to smallest (use np.sort)\n",
+ " # TODO -- replace this line\n",
+ " sorted_prob_over_tokens = prob_over_tokens\n",
+ "\n",
+ " # Find the probability at the k'th position\n",
+ " # TODO -- replace this line\n",
+ " kth_prob_value = 0.0\n",
+ "\n",
+ " # Set all probabilities below this value to zero\n",
+ " prob_over_tokens[prob_over_tokens"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 13.1: Graph representation**\n",
+ "\n",
+ "This notebook investigates representing graphs with matrices as illustrated in figure 13.4 from the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "import networkx as nx"
+ ],
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Routine to draw graph structure\n",
+ "def draw_graph_structure(adjacency_matrix):\n",
+ "\n",
+ " G = nx.Graph()\n",
+ " n_node = adjacency_matrix.shape[0]\n",
+ " for i in range(n_node):\n",
+ " for j in range(i):\n",
+ " if adjacency_matrix[i,j]:\n",
+ " G.add_edge(i,j)\n",
+ "\n",
+ " nx.draw(G, nx.spring_layout(G, seed = 0), with_labels=True)\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "O1QMxC7X4vh9"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define a graph\n",
+ "# Note that the nodes are labelled from 0 rather than 1 as in the book\n",
+ "A = np.array([[0,1,0,1,0,0,0,0],\n",
+ " [1,0,1,1,1,0,0,0],\n",
+ " [0,1,0,0,1,0,0,0],\n",
+ " [1,1,0,0,1,0,0,0],\n",
+ " [0,1,1,1,0,1,0,1],\n",
+ " [0,0,0,0,1,0,1,1],\n",
+ " [0,0,0,0,0,1,0,0],\n",
+ " [0,0,0,0,1,1,0,0]]);\n",
+ "print(A)\n",
+ "draw_graph_structure(A)"
+ ],
+ "metadata": {
+ "id": "TIrihEw-7DRV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO -- find algorithmically how many walks of length three are between nodes 3 and 7\n",
+ "# Replace this line\n",
+ "print(\"Number of walks between nodes three and seven = ???\")"
+ ],
+ "metadata": {
+ "id": "PzvfUpkV4zCj"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO -- find algorithmically what the minimum path distance between nodes 0 and 6 is\n",
+ "# (i.e. what is the first walk length with non-zero count between 0 and 6)\n",
+ "# Replace this line\n",
+ "print(\"Minimum distance = ???\")\n",
+ "\n",
+ "\n",
+ "# What is the worst case complexity of your method?"
+ ],
+ "metadata": {
+ "id": "MhhJr6CgCRb5"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now let's represent node 0 as a vector\n",
+ "x = np.array([[1],[0],[0],[0],[0],[0],[0],[0]]);\n",
+ "print(x)"
+ ],
+ "metadata": {
+ "id": "lCQjXlatABGZ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO: Find algorithmically how many paths of length 3 are there between node 0 and every other node\n",
+ "# Replace this line\n",
+ "print(np.zeros_like(x))"
+ ],
+ "metadata": {
+ "id": "nizLdZgLDzL4"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap13/13_2_Graph_Classification.ipynb b/Notebooks/Chap13/13_2_Graph_Classification.ipynb
new file mode 100644
index 0000000..b5d2fd9
--- /dev/null
+++ b/Notebooks/Chap13/13_2_Graph_Classification.ipynb
@@ -0,0 +1,244 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyOMSGUFWT+YN0fwYHpMmHJM",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 13.2: Graph classification**\n",
+ "\n",
+ "This notebook investigates representing graphs with matrices as illustrated in figure 13.4 from the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "import networkx as nx"
+ ],
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's build a model that maps a chemical structure to a binary decision. This model might be used to predict whether a chemical is liquid at room temperature or not. We'll start by drawing the chemical structure."
+ ],
+ "metadata": {
+ "id": "UNleESc7k5uB"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define a graph that represents the chemical structure of ethanol and draw it\n",
+ "# Each node is labelled with the node number and the element (carbon, hydrogen, oxygen)\n",
+ "G = nx.Graph()\n",
+ "G.add_edge('0:H','2:C')\n",
+ "G.add_edge('1:H','2:C')\n",
+ "G.add_edge('3:H','2:C')\n",
+ "G.add_edge('2:C','5:C')\n",
+ "G.add_edge('4:H','5:C')\n",
+ "G.add_edge('6:H','5:C')\n",
+ "G.add_edge('7:O','5:C')\n",
+ "G.add_edge('8:H','7:O')\n",
+ "nx.draw(G, nx.spring_layout(G, seed = 0), with_labels=True, node_size=600)\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "TIrihEw-7DRV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define adjacency matrix\n",
+ "# TODO -- Define the adjacency matrix for this chemical\n",
+ "# Replace this line\n",
+ "A = np.zeros((9,9)) ;\n",
+ "\n",
+ "\n",
+ "print(A)\n",
+ "\n",
+ "# TODO -- Define node matrix\n",
+ "# There will be 9 nodes and 118 possible chemical elements\n",
+ "# so we'll define a 9x118 matrix. Each column represents one\n",
+ "# node and is a one-hot vector (i.e. all zeros, except a single one at the\n",
+ "# chemical number of the element).\n",
+ "# Chemical numbers: Hydrogen-->1, Carbon-->6, Oxygen-->8\n",
+ "# Since the indices start at 0, we'll set element 0 to 1 for hydrogen, element 5\n",
+ "# to one for carbon, and element 7 to one for oxygen\n",
+ "# Replace this line:\n",
+ "X = np.zeros((118,9))\n",
+ "\n",
+ "\n",
+ "# Print the top 15 rows of the data matrix\n",
+ "print(X[0:15,:])"
+ ],
+ "metadata": {
+ "id": "gKBD5JsPfrkA"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's define a network with four layers that maps this graph to a binary value, using the formulation in equation 13.11."
+ ],
+ "metadata": {
+ "id": "40FLjNIcpHa9"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# We'll need these helper functions\n",
+ "\n",
+ "# Define the Rectified Linear Unit (ReLU) function\n",
+ "def ReLU(preactivation):\n",
+ " activation = preactivation.clip(0.0)\n",
+ " return activation\n",
+ "\n",
+ "# Define the logistic sigmoid function\n",
+ "def sigmoid(x):\n",
+ " return 1.0/(1.0+np.exp(-x))"
+ ],
+ "metadata": {
+ "id": "52IFREpepHE4"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Our network will have K=3 hidden layers, and will use a dimension of D=200.\n",
+ "K = 3; D = 200\n",
+ "# Set seed so we always get the same random numbers\n",
+ "np.random.seed(1)\n",
+ "# Let's initialize the parameter matrices randomly with He initialization\n",
+ "Omega0 = np.random.normal(size=(D, 118)) * 2.0 / D\n",
+ "beta0 = np.random.normal(size=(D,1)) * 2.0 / D\n",
+ "Omega1 = np.random.normal(size=(D, D)) * 2.0 / D\n",
+ "beta1 = np.random.normal(size=(D,1)) * 2.0 / D\n",
+ "Omega2 = np.random.normal(size=(D, D)) * 2.0 / D\n",
+ "beta2 = np.random.normal(size=(D,1)) * 2.0 / D\n",
+ "omega3 = np.random.normal(size=(1, D))\n",
+ "beta3 = np.random.normal(size=(1,1))"
+ ],
+ "metadata": {
+ "id": "ag0YdEgnpApK"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def graph_neural_network(A,X, Omega0, beta0, Omega1, beta1, Omega2, beta2, omega3, beta3):\n",
+ " # Define this network according to equation 13.11 from the book\n",
+ " # Replace this line\n",
+ " f = np.ones((1,1))\n",
+ "\n",
+ " return f;"
+ ],
+ "metadata": {
+ "id": "RQuTMc2WrsU3"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's test this network\n",
+ "f = graph_neural_network(A,X, Omega0, beta0, Omega1, beta1, Omega2, beta2, omega3, beta3)\n",
+ "print(\"Your value is %3f: \"%(f[0,0]), \"True value of f: 0.310843\")"
+ ],
+ "metadata": {
+ "id": "X7gYgOu6uIAt"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's check that permuting the indices of the graph doesn't change\n",
+ "# the output of the network\n",
+ "# Define a permutation matrix\n",
+ "P = np.array([[0,1,0,0,0,0,0,0,0],\n",
+ " [0,0,0,0,1,0,0,0,0],\n",
+ " [0,0,0,0,0,1,0,0,0],\n",
+ " [0,0,0,0,0,0,0,0,1],\n",
+ " [1,0,0,0,0,0,0,0,0],\n",
+ " [0,0,1,0,0,0,0,0,0],\n",
+ " [0,0,0,1,0,0,0,0,0],\n",
+ " [0,0,0,0,0,0,0,1,0],\n",
+ " [0,0,0,0,0,0,1,0,0]]);\n",
+ "\n",
+ "# TODO -- Use this matrix to permute the adjacency matrix A and node matrix X\n",
+ "# Replace these lines\n",
+ "A_permuted = np.copy(A)\n",
+ "X_permuted = np.copy(X)\n",
+ "\n",
+ "f = graph_neural_network(A_permuted,X_permuted, Omega0, beta0, Omega1, beta1, Omega2, beta2, omega3, beta3)\n",
+ "print(\"Your value is %3f: \"%(f[0,0]), \"True value of f: 0.310843\")"
+ ],
+ "metadata": {
+ "id": "F0zc3U_UuR5K"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "TODO -- encode the adjacency matrix and node matrix for propanol and run the network again. Show that the network still runs even though the size of the input graph is different.\n",
+ "\n",
+ "Propanol structure can be found [here](https://upload.wikimedia.org/wikipedia/commons/b/b8/Propanol_flat_structure.png)."
+ ],
+ "metadata": {
+ "id": "l44vHi50zGqY"
+ }
+ }
+ ]
+}
diff --git a/Notebooks/Chap13/13_3_Neighborhood_Sampling.ipynb b/Notebooks/Chap13/13_3_Neighborhood_Sampling.ipynb
new file mode 100644
index 0000000..3a0bc94
--- /dev/null
+++ b/Notebooks/Chap13/13_3_Neighborhood_Sampling.ipynb
@@ -0,0 +1,314 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyNXqwmC4yEc1mGv9/74b0jY",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 13.3: Neighborhood sampling**\n",
+ "\n",
+ "This notebook investigates neighborhood sampling of graphs as in figure 13.10 from the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "import networkx as nx"
+ ],
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's construct the graph from figure 13.10, which has 23 nodes."
+ ],
+ "metadata": {
+ "id": "UNleESc7k5uB"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define adjacency matrix\n",
+ "A = np.array([[0,1,1,1,0, 0,0,0,0,0, 0,0,0,0,0, 0,0,0,0,0, 0,0,0],\n",
+ " [1,0,1,0,0, 0,0,0,1,1, 0,0,0,0,0, 0,0,0,0,0, 0,0,0],\n",
+ " [1,1,0,1,0, 0,0,0,0,1, 0,0,0,0,0, 0,0,0,0,0, 0,0,0],\n",
+ " [1,0,1,0,1, 0,1,1,0,0, 0,0,0,0,0, 0,0,0,0,0, 0,0,0],\n",
+ " [0,0,0,1,0, 1,0,1,0,0, 0,0,0,0,0, 0,0,0,0,0, 0,0,0],\n",
+ " [0,0,0,0,1, 0,0,1,0,0, 0,0,0,0,0, 0,0,0,0,0, 0,0,0],\n",
+ " [0,0,0,1,0, 0,0,1,0,1, 1,0,0,0,0, 0,0,0,0,0, 0,0,0],\n",
+ " [0,0,0,1,1, 1,1,0,0,0, 1,0,0,1,0, 0,0,0,0,0, 0,0,0],\n",
+ " [0,1,0,0,0, 0,0,0,0,1, 0,0,0,0,0, 0,0,0,0,0, 0,0,0],\n",
+ " [0,1,1,0,0, 0,1,0,1,0, 0,1,1,0,0, 0,1,0,0,0, 0,0,0],\n",
+ " [0,0,0,0,0, 0,1,1,0,0, 0,0,1,0,0, 0,0,0,0,0, 0,0,0],\n",
+ " [0,0,0,0,0, 0,0,0,0,1, 0,0,0,0,1, 1,1,0,0,0, 0,0,0],\n",
+ " [0,0,0,0,0, 0,0,0,0,1, 1,0,0,1,0, 0,1,1,0,0, 0,0,0],\n",
+ " [0,0,0,0,0, 0,0,1,0,0, 0,0,1,0,0, 0,0,1,1,0, 0,0,0],\n",
+ " [0,0,0,0,0, 0,0,0,0,0, 0,1,0,0,0, 1,0,0,0,1, 0,0,0],\n",
+ " [0,0,0,0,0, 0,0,0,0,0, 0,1,0,0,1, 0,1,0,0,1, 0,0,0],\n",
+ " [0,0,0,0,0, 0,0,0,0,1, 0,1,1,0,0, 1,0,1,0,1, 0,0,0],\n",
+ " [0,0,0,0,0, 0,0,0,0,0, 0,0,1,1,0, 0,1,0,1,0, 1,1,1],\n",
+ " [0,0,0,0,0, 0,0,0,0,0, 0,0,0,1,0, 0,0,1,0,0, 0,0,1],\n",
+ " [0,0,0,0,0, 0,0,0,0,0, 0,0,0,0,1, 1,1,0,0,0, 1,0,0],\n",
+ " [0,0,0,0,0, 0,0,0,0,0, 0,0,0,0,0, 0,0,1,0,1, 0,1,0],\n",
+ " [0,0,0,0,0, 0,0,0,0,0, 0,0,0,0,0, 0,0,1,0,0, 1,0,1],\n",
+ " [0,0,0,0,0, 0,0,0,0,0, 0,0,0,0,0, 0,0,1,1,0, 0,1,0]]);\n",
+ "print(A)"
+ ],
+ "metadata": {
+ "id": "fHgH5hdG_W1h"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Routine to draw graph structure, highlighting original node (brown in fig 13.10)\n",
+ "# and neighborhood nodes (orange in figure 13.10)\n",
+ "def draw_graph_structure(adjacency_matrix, original_node, neighborhood_nodes=None):\n",
+ "\n",
+ " G = nx.Graph()\n",
+ " n_node = adjacency_matrix.shape[0]\n",
+ " for i in range(n_node):\n",
+ " for j in range(i):\n",
+ " if adjacency_matrix[i,j]:\n",
+ " G.add_edge(i,j)\n",
+ "\n",
+ " color_map = []\n",
+ "\n",
+ " for node in G:\n",
+ " if original_node[node]:\n",
+ " color_map.append('brown')\n",
+ " else:\n",
+ " if neighborhood_nodes[node]:\n",
+ " color_map.append('orange')\n",
+ " else:\n",
+ " color_map.append('white')\n",
+ "\n",
+ " nx.draw(G, nx.spring_layout(G, seed = 7), with_labels=True,node_color=color_map)\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "TIrihEw-7DRV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "n_nodes = A.shape[0]\n",
+ "\n",
+ "# Define a single output layer node\n",
+ "output_layer_nodes=np.zeros((n_nodes,1)); output_layer_nodes[16]=1\n",
+ "# Define the neighboring nodes to draw (none)\n",
+ "neighbor_nodes = np.zeros((n_nodes,1))\n",
+ "print(\"Output layer:\")\n",
+ "draw_graph_structure(A, output_layer_nodes, neighbor_nodes)"
+ ],
+ "metadata": {
+ "id": "gKBD5JsPfrkA"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Let's imagine that we want to form a batch for a node labelling task that consists of just node 16 in the output layer (highlighted). The network consists of the input, hidden layer 1, hidden layer2, and the output layer."
+ ],
+ "metadata": {
+ "id": "JaH3g_-O-0no"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO Find the nodes in hidden layer 2 that connect to node 16 in the output layer\n",
+ "# using the adjacency matrix\n",
+ "# Replace this line:\n",
+ "hidden_layer2_nodes = np.zeros((n_nodes,1));\n",
+ "\n",
+ "print(\"Hidden layer 2:\")\n",
+ "draw_graph_structure(A, output_layer_nodes, hidden_layer2_nodes)"
+ ],
+ "metadata": {
+ "id": "9oSiuP3B3HNS"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO - Find the nodes in hidden layer 1 that connect that connect to node 16 in the output layer\n",
+ "# using the adjacency matrix\n",
+ "# Replace this line:\n",
+ "hidden_layer1_nodes = np.zeros((n_nodes,1));\n",
+ "\n",
+ "print(\"Hidden layer 1:\")\n",
+ "draw_graph_structure(A, output_layer_nodes, hidden_layer1_nodes)"
+ ],
+ "metadata": {
+ "id": "zZFxw3m1_wWr"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO Find the nodes in the input layer that connect to node 16 in the output layer\n",
+ "# using the adjacency matrix\n",
+ "# Replace this line:\n",
+ "input_layer_nodes = np.zeros((n_nodes,1));\n",
+ "\n",
+ "print(\"Input layer:\")\n",
+ "draw_graph_structure(A, output_layer_nodes, input_layer_nodes)"
+ ],
+ "metadata": {
+ "id": "EL3N8BXyCu0F"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "This is bad news. This is a fairly sparsely connected graph (i.e. adjacency matrix is mostly zeros) and there are only two hidden layers. Nonetheless, we have to involve almost all the nodes in the graph to compute the loss at this output.\n",
+ "\n",
+ "To resolve this problem, we'll use neighborhood sampling. We'll start again with a single node in the output layer."
+ ],
+ "metadata": {
+ "id": "CE0WqytvC7zr"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "print(\"Output layer:\")\n",
+ "draw_graph_structure(A, output_layer_nodes, neighbor_nodes)"
+ ],
+ "metadata": {
+ "id": "59WNys3KC5y6"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define umber of neighbors to sample\n",
+ "n_sample = 3"
+ ],
+ "metadata": {
+ "id": "uCoJwpcTNFdI"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO Find the nodes in hidden layer 2 that connect to node 16 in the output layer\n",
+ "# using the adjacency matrix. Then sample n_sample of these nodes randomly without\n",
+ "# replacement.\n",
+ "\n",
+ "# Replace this line:\n",
+ "hidden_layer2_nodes = np.zeros((n_nodes,1));\n",
+ "\n",
+ "draw_graph_structure(A, output_layer_nodes, hidden_layer2_nodes)"
+ ],
+ "metadata": {
+ "id": "_WEop6lYGNhJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO Find the nodes in hidden layer 1 that connect to the nodes in hidden layer 2\n",
+ "# using the adjacency matrix. Then sample n_sample of these nodes randomly without\n",
+ "# replacement. Make sure not to sample nodes that were already included in hidden layer 2 our the output layer.\n",
+ "# The nodes at hidden layer 1 are the union of these nodes and the nodes in hidden layer 2\n",
+ "\n",
+ "# Replace this line:\n",
+ "hidden_layer1_nodes = np.zeros((n_nodes,1));\n",
+ "\n",
+ "draw_graph_structure(A, output_layer_nodes, hidden_layer1_nodes)\n"
+ ],
+ "metadata": {
+ "id": "k90qW_LDLpNk"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO Find the nodes in the input layer that connect to the nodes in hidden layer 1\n",
+ "# using the adjacency matrix. Then sample n_sample of these nodes randomly without\n",
+ "# replacement. Make sure not to sample nodes that were already included in hidden layer 1,2, or the output layer.\n",
+ "# The nodes at the input layer are the union of these nodes and the nodes in hidden layers 1 and 2\n",
+ "\n",
+ "# Replace this line:\n",
+ "input_layer_nodes = np.zeros((n_nodes,1));\n",
+ "\n",
+ "draw_graph_structure(A, output_layer_nodes, input_layer_nodes)"
+ ],
+ "metadata": {
+ "id": "NDEYUty_O3Zr"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "If you did this correctly, there should be 9 yellow nodes in the figure. The \"receptive field\" of node 16 in the output layer increases much more slowly as we move back through the layers of the network."
+ ],
+ "metadata": {
+ "id": "vu4eJURmVkc5"
+ }
+ }
+ ]
+}
diff --git a/Notebooks/Chap13/13_4_Graph_Attention_Networks.ipynb b/Notebooks/Chap13/13_4_Graph_Attention_Networks.ipynb
new file mode 100644
index 0000000..7c33404
--- /dev/null
+++ b/Notebooks/Chap13/13_4_Graph_Attention_Networks.ipynb
@@ -0,0 +1,213 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyO/wJ4N9w01f04mmrs/ZSHY",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 13.4: Graph attention networks**\n",
+ "\n",
+ "This notebook builds a graph attention mechanism from scratch, as discussed in section 13.8.6 of the book and illustrated in figure 13.12c\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ],
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The self-attention mechanism maps $N$ inputs $\\mathbf{x}_{n}\\in\\mathbb{R}^{D}$ and returns $N$ outputs $\\mathbf{x}'_{n}\\in \\mathbb{R}^{D}$. \n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "9OJkkoNqCVK2"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set seed so we get the same random numbers\n",
+ "np.random.seed(1)\n",
+ "# Number of nodes in the graph\n",
+ "N = 8\n",
+ "# Number of dimensions of each input\n",
+ "D = 4\n",
+ "\n",
+ "# Define a graph\n",
+ "A = np.array([[0,1,0,1,0,0,0,0],\n",
+ " [1,0,1,1,1,0,0,0],\n",
+ " [0,1,0,0,1,0,0,0],\n",
+ " [1,1,0,0,1,0,0,0],\n",
+ " [0,1,1,1,0,1,0,1],\n",
+ " [0,0,0,0,1,0,1,1],\n",
+ " [0,0,0,0,0,1,0,0],\n",
+ " [0,0,0,0,1,1,0,0]]);\n",
+ "print(A)\n",
+ "\n",
+ "# Let's also define some random data\n",
+ "X = np.random.normal(size=(D,N))"
+ ],
+ "metadata": {
+ "id": "oAygJwLiCSri"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "We'll also need the weights and biases for the keys, queries, and values (equations 12.2 and 12.4)"
+ ],
+ "metadata": {
+ "id": "W2iHFbtKMaDp"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Choose random values for the parameters\n",
+ "omega = np.random.normal(size=(D,D))\n",
+ "beta = np.random.normal(size=(D,1))\n",
+ "phi = np.random.normal(size=(1,2*D))"
+ ],
+ "metadata": {
+ "id": "79TSK7oLMobe"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "We'll need a softmax operation that operates on the columns of the matrix and a ReLU function as well"
+ ],
+ "metadata": {
+ "id": "iYPf6c4MhCgq"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define softmax operation that works independently on each column\n",
+ "def softmax_cols(data_in):\n",
+ " # Exponentiate all of the values\n",
+ " exp_values = np.exp(data_in) ;\n",
+ " # Sum over columns\n",
+ " denom = np.sum(exp_values, axis = 0);\n",
+ " # Replicate denominator to N rows\n",
+ " denom = np.matmul(np.ones((data_in.shape[0],1)), denom[np.newaxis,:])\n",
+ " # Compute softmax\n",
+ " softmax = exp_values / denom\n",
+ " # return the answer\n",
+ " return softmax\n",
+ "\n",
+ "\n",
+ "# Define the Rectified Linear Unit (ReLU) function\n",
+ "def ReLU(preactivation):\n",
+ " activation = preactivation.clip(0.0)\n",
+ " return activation\n"
+ ],
+ "metadata": {
+ "id": "obaQBdUAMXXv"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ " # Now let's compute self attention in matrix form\n",
+ "def graph_attention(X,omega, beta, phi, A):\n",
+ "\n",
+ " # TODO -- Write this function (see figure 13.12c)\n",
+ " # 1. Compute X_prime\n",
+ " # 2. Compute S\n",
+ " # 3. To apply the mask, set S to a very large negative number (e.g. -1e20) everywhere where A+I is zero\n",
+ " # 4. Run the softmax function to compute the attention values\n",
+ " # 5. Postmultiply X' by the attention values\n",
+ " # 6. Apply the ReLU function\n",
+ " # Replace this line:\n",
+ " output = np.ones_like(X) ;\n",
+ "\n",
+ " return output;"
+ ],
+ "metadata": {
+ "id": "gb2WvQ3SiH8r"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Test out the graph attention mechanism\n",
+ "np.set_printoptions(precision=3)\n",
+ "output = graph_attention(X, omega, beta, phi, A);\n",
+ "print(\"Correct answer is:\")\n",
+ "print(\"[[0. 0.028 0.37 0. 0.97 0. 0. 0.698]\")\n",
+ "print(\" [0. 0. 0. 0. 1.184 0. 2.654 0. ]\")\n",
+ "print(\" [1.13 0.564 0. 1.298 0.268 0. 0. 0.779]\")\n",
+ "print(\" [0.825 0. 0. 1.175 0. 0. 0. 0. ]]]\")\n",
+ "\n",
+ "\n",
+ "print(\"Your answer is:\")\n",
+ "print(output)"
+ ],
+ "metadata": {
+ "id": "d4p6HyHXmDh5"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "TODO -- Try to construct a dot-product self-attention mechanism as in practical 12.1 that respects the geometry of the graph and has zero attention between non-neighboring nodes by combining figures 13.12a and 13.12b.\n"
+ ],
+ "metadata": {
+ "id": "QDEkIrcgrql-"
+ }
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap15/15_1_GAN_Toy_Example.ipynb b/Notebooks/Chap15/15_1_GAN_Toy_Example.ipynb
new file mode 100644
index 0000000..4f18380
--- /dev/null
+++ b/Notebooks/Chap15/15_1_GAN_Toy_Example.ipynb
@@ -0,0 +1,419 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyM0StKV3FIZ3MZqfflqC0Rv",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 15.1: GAN Toy example**\n",
+ "\n",
+ "This notebook investigates the GAN toy example as illustrated in figure 15.1 in the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ],
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Get a batch of real data. Our goal is to make data that looks like this.\n",
+ "def get_real_data_batch(n_sample):\n",
+ " np.random.seed(0)\n",
+ " x_true = np.random.normal(size=(1,n_sample)) + 7.5\n",
+ " return x_true"
+ ],
+ "metadata": {
+ "id": "y_OkVWmam4Qx"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define our generator. This takes a standard normally-distributed latent variable $z$ and adds a scalar $\\theta$ to this, where $\\theta$ is the single parameter of this generative model according to:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "x_i = z_i + \\theta.\n",
+ "\\end{equation}\n",
+ "\n",
+ "Obviously this model can generate the family of Gaussian distributions with unit variance, but different means."
+ ],
+ "metadata": {
+ "id": "RFpL0uCXoTpV"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# This is our generator -- takes the single parameter theta\n",
+ "# of the generative model and generates n samples\n",
+ "def generator(z, theta):\n",
+ " x_gen = z + theta\n",
+ " return x_gen"
+ ],
+ "metadata": {
+ "id": "OtLQvf3Enfyw"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now, we define our discriminator. This is a simple logistic regression model (1D linear model passed through sigmoid) that returns the probability that the data is real"
+ ],
+ "metadata": {
+ "id": "Xrzd8aehYAYR"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define our discriminative model\n",
+ "\n",
+ "# Logistic sigmoid, maps from [-infty,infty] to [0,1]\n",
+ "def sig(data_in):\n",
+ " return 1.0 / (1.0+np.exp(-data_in))\n",
+ "\n",
+ "# Discriminator computes y\n",
+ "def discriminator(x, phi0, phi1):\n",
+ " return sig(phi0 + phi1 * x)"
+ ],
+ "metadata": {
+ "id": "vHBgAFZMsnaC"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Draws a figure like Figure 15.1a\n",
+ "def draw_data_model(x_real, x_syn, phi0=None, phi1=None):\n",
+ " fix, ax = plt.subplots();\n",
+ "\n",
+ " for x in x_syn:\n",
+ " ax.plot([x,x],[0,0.33],color='#f47a60')\n",
+ " for x in x_real:\n",
+ " ax.plot([x,x],[0,0.33],color='#7fe7dc')\n",
+ "\n",
+ " if phi0 is not None:\n",
+ " x_model = np.arange(0,10,0.01)\n",
+ " y_model = discriminator(x_model, phi0, phi1)\n",
+ " ax.plot(x_model, y_model,color='#dddddd')\n",
+ " ax.set_xlim([0,10])\n",
+ " ax.set_ylim([0,1])\n",
+ "\n",
+ "\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "V1FiDBhepcQJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Get data batch\n",
+ "x_real = get_real_data_batch(10)\n",
+ "\n",
+ "# Initialize generator and synthesize a batch of examples\n",
+ "theta = 3.0\n",
+ "np.random.seed(1)\n",
+ "z = np.random.normal(size=(1,10))\n",
+ "x_syn = generator(z, theta)\n",
+ "\n",
+ "# Initialize discriminator model\n",
+ "phi0 = -2\n",
+ "phi1 = 1\n",
+ "\n",
+ "draw_data_model(x_real, x_syn, phi0, phi1)"
+ ],
+ "metadata": {
+ "id": "U8pFb497x36n"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "You can see that the synthesized (orange) samples don't look much like the real (cyan) ones, and the initial model to discriminate them (gray line represents probability of being real) is pretty bad as well.\n",
+ "\n",
+ "Let's deal with the discriminator first. Let's define the loss"
+ ],
+ "metadata": {
+ "id": "SNDV1G5PYhcQ"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Discriminator loss\n",
+ "def compute_discriminator_loss(x_real, x_syn, phi0, phi1):\n",
+ "\n",
+ " # TODO -- compute the loss for the discriminator\n",
+ " # Run the real data and the synthetic data through the discriminator\n",
+ " # Then use the standard binary cross entropy loss with the y=1 for the real samples\n",
+ " # and y=0 for the synthesized ones.\n",
+ " # Replace this line\n",
+ " loss = 0.0\n",
+ "\n",
+ "\n",
+ " return loss"
+ ],
+ "metadata": {
+ "id": "Bc3VwCabYcfg"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Test the loss\n",
+ "loss = compute_discriminator_loss(x_real, x_syn, phi0, phi1)\n",
+ "print(\"True Loss = 13.814757170851447, Your loss=\", loss )"
+ ],
+ "metadata": {
+ "id": "MiqM3GXSbn0z"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Gradient of loss (cheating, using finite differences)\n",
+ "def compute_discriminator_gradient(x_real, x_syn, phi0, phi1):\n",
+ " delta = 0.0001;\n",
+ " loss1 = compute_discriminator_loss(x_real, x_syn, phi0, phi1)\n",
+ " loss2 = compute_discriminator_loss(x_real, x_syn, phi0+delta, phi1)\n",
+ " loss3 = compute_discriminator_loss(x_real, x_syn, phi0, phi1+delta)\n",
+ " dl_dphi0 = (loss2-loss1) / delta\n",
+ " dl_dphi1 = (loss3-loss1) / delta\n",
+ "\n",
+ " return dl_dphi0, dl_dphi1\n",
+ "\n",
+ "# This routine performs gradient descent with the discriminator\n",
+ "def update_discriminator(x_real, x_syn, n_iter, phi0, phi1):\n",
+ "\n",
+ " # Define learning rate\n",
+ " alpha = 0.01\n",
+ "\n",
+ " # Get derivatives\n",
+ " print(\"Initial discriminator loss = \", compute_discriminator_loss(x_real, x_syn, phi0, phi1))\n",
+ " for iter in range(n_iter):\n",
+ " # Get gradient\n",
+ " dl_dphi0, dl_dphi1 = compute_discriminator_gradient(x_real, x_syn, phi0, phi1)\n",
+ " # Take a gradient step downhill\n",
+ " phi0 = phi0 - alpha * dl_dphi0 ;\n",
+ " phi1 = phi1 - alpha * dl_dphi1 ;\n",
+ "\n",
+ " print(\"Final Discriminator Loss= \", compute_discriminator_loss(x_real, x_syn, phi0, phi1))\n",
+ "\n",
+ " return phi0, phi1"
+ ],
+ "metadata": {
+ "id": "zAxUPo3p0CIW"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's update the discriminator (sigmoid curve)\n",
+ "n_iter = 100\n",
+ "print(\"Initial parameters (phi0,phi1)\", phi0, phi1)\n",
+ "phi0, phi1 = update_discriminator(x_real, x_syn, n_iter, phi0, phi1)\n",
+ "print(\"Final parameters (phi0,phi1)\", phi0, phi1)\n",
+ "draw_data_model(x_real, x_syn, phi0, phi1)"
+ ],
+ "metadata": {
+ "id": "FE_DeweeAbMc"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's update the generator"
+ ],
+ "metadata": {
+ "id": "pRv9myh0d3Xm"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def compute_generator_loss(z, theta, phi0, phi1):\n",
+ " # TODO -- Run the generator on the latent variables z with the parameters theta\n",
+ " # to generate new data x_syn\n",
+ " # Then run the discriminator on the new data to get the probability of being real\n",
+ " # The loss is the total negative log probability of being synthesized (i.e. of not being real)\n",
+ " # Replace this code\n",
+ " loss = 1\n",
+ "\n",
+ "\n",
+ " return loss"
+ ],
+ "metadata": {
+ "id": "5uiLrFBvJFAr"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Test generator loss to check you have it correct\n",
+ "loss = compute_generator_loss(z, theta, -2, 1)\n",
+ "print(\"True Loss = 13.78437035945412, Your loss=\", loss )"
+ ],
+ "metadata": {
+ "id": "cqnU3dGPd6NK"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def compute_generator_gradient(z, theta, phi0, phi1):\n",
+ " delta = 0.0001\n",
+ " loss1 = compute_generator_loss(z,theta, phi0, phi1) ;\n",
+ " loss2 = compute_generator_loss(z,theta+delta, phi0, phi1) ;\n",
+ " dl_dtheta = (loss2-loss1)/ delta\n",
+ " return dl_dtheta\n",
+ "\n",
+ "def update_generator(z, theta, n_iter, phi0, phi1):\n",
+ " # Define learning rate\n",
+ " alpha = 0.02\n",
+ "\n",
+ " # Get derivatives\n",
+ " print(\"Initial generator loss = \", compute_generator_loss(z, theta, phi0, phi1))\n",
+ " for iter in range(n_iter):\n",
+ " # Get gradient\n",
+ " dl_dtheta = compute_generator_gradient(x_real, x_syn, phi0, phi1)\n",
+ " # Take a gradient step (uphill, since we are trying to make synthesized data less well classified by discriminator)\n",
+ " theta = theta + alpha * dl_dtheta ;\n",
+ "\n",
+ " print(\"Final generator loss = \", compute_generator_loss(z, theta, phi0, phi1))\n",
+ " return theta\n"
+ ],
+ "metadata": {
+ "id": "P1Lqy922dqal"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "n_iter = 10\n",
+ "theta = 3.0\n",
+ "print(\"Theta before\", theta)\n",
+ "theta = update_generator(z, theta, n_iter, phi0, phi1)\n",
+ "print(\"Theta after\", theta)\n",
+ "\n",
+ "x_syn = generator(z,theta)\n",
+ "draw_data_model(x_real, x_syn, phi0, phi1)"
+ ],
+ "metadata": {
+ "id": "Q6kUkMO1P8V0"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now let's define a full GAN loop\n",
+ "\n",
+ "# Initialize the parameters\n",
+ "theta = 3\n",
+ "phi0 = -2\n",
+ "phi1 = 1\n",
+ "\n",
+ "# Number of iterations for updating generator and discriminator\n",
+ "n_iter_discrim = 300\n",
+ "n_iter_gen = 3\n",
+ "\n",
+ "print(\"Final parameters (phi0,phi1)\", phi0, phi1)\n",
+ "for c_gan_iter in range(5):\n",
+ "\n",
+ " # Run generator to product synthesized data\n",
+ " x_syn = generator(z, theta)\n",
+ " draw_data_model(x_real, x_syn, phi0, phi1)\n",
+ "\n",
+ " # Update the discriminator\n",
+ " print(\"Updating discriminator\")\n",
+ " phi0, phi1 = update_discriminator(x_real, x_syn, n_iter_discrim, phi0, phi1)\n",
+ " draw_data_model(x_real, x_syn, phi0, phi1)\n",
+ "\n",
+ " # Update the generator\n",
+ " print(\"Updating generator\")\n",
+ " theta = update_generator(z, theta, n_iter_gen, phi0, phi1)\n"
+ ],
+ "metadata": {
+ "id": "pcbdK2agTO-y"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "You can see that the synthesized data (orange) is becoming closer to the true data (cyan). However, this is extremely unstable -- as you will find if you mess around with the number of iterations of each optimization and the total iterations overall."
+ ],
+ "metadata": {
+ "id": "loMx0TQUgBs7"
+ }
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap15/15_2_Wasserstein_Distance.ipynb b/Notebooks/Chap15/15_2_Wasserstein_Distance.ipynb
new file mode 100644
index 0000000..b0561b6
--- /dev/null
+++ b/Notebooks/Chap15/15_2_Wasserstein_Distance.ipynb
@@ -0,0 +1,245 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 15.2: Wasserstein Distance**\n",
+ "\n",
+ "This notebook investigates the GAN toy example as illustrated in figure 15.1 in the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from matplotlib import cm\n",
+ "from matplotlib.colors import ListedColormap\n",
+ "from scipy.optimize import linprog"
+ ],
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define two probability distributions\n",
+ "p = np.array([5, 3, 2, 1, 8, 7, 5, 9, 2, 1])\n",
+ "q = np.array([4, 10,1, 1, 4, 6, 3, 2, 0, 1])\n",
+ "p = p/np.sum(p);\n",
+ "q= q/np.sum(q);\n",
+ "\n",
+ "# Draw those distributions\n",
+ "fig, ax =plt.subplots(2,1);\n",
+ "x = np.arange(0,p.size,1)\n",
+ "ax[0].bar(x,p, color=\"#cccccc\")\n",
+ "ax[0].set_ylim([0,0.35])\n",
+ "ax[0].set_ylabel(\"p(x=i)\")\n",
+ "\n",
+ "ax[1].bar(x,q,color=\"#f47a60\")\n",
+ "ax[1].set_ylim([0,0.35])\n",
+ "ax[1].set_ylabel(\"q(x=j)\")\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "ZIfQwhd-AV6L"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO Define the distance matrix from figure 15.8d\n",
+ "# Replace this line\n",
+ "dist_mat = np.zeros((10,10))\n",
+ "\n",
+ "# vectorize the distance matrix\n",
+ "c = dist_mat.flatten()"
+ ],
+ "metadata": {
+ "id": "EZSlZQzWBKTm"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define pretty colormap\n",
+ "my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ "my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ "r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ "g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ "b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ "my_colormap = ListedColormap(np.vstack((r,g,b)).transpose()/255.0)\n",
+ "\n",
+ "def draw_2D_heatmap(data, title, my_colormap):\n",
+ " # Make grid of intercept/slope values to plot\n",
+ " xv, yv = np.meshgrid(np.linspace(0, 1, 10), np.linspace(0, 1, 10))\n",
+ " fig,ax = plt.subplots()\n",
+ " fig.set_size_inches(4,4)\n",
+ " plt.imshow(data, cmap=my_colormap)\n",
+ " ax.set_title(title)\n",
+ " ax.set_xlabel('$q$'); ax.set_ylabel('$p$')\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "ABRANmp6F8iQ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "draw_2D_heatmap(dist_mat,'Distance $|i-j|$', my_colormap)"
+ ],
+ "metadata": {
+ "id": "G0HFPBXyHT6V"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define b to be the verticalconcatenation of p and q\n",
+ "b = np.hstack((p,q))[np.newaxis].transpose()"
+ ],
+ "metadata": {
+ "id": "SfqeT3KlHWrt"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO: Now construct the matrix A that has the initial distribution constraints\n",
+ "# so that A @ TPFlat=b where TPFlat is the transport plan TP vectorized rows first so TPFlat = np.flatten(TP)\n",
+ "# Replace this line:\n",
+ "A = np.zeros((20,100))"
+ ],
+ "metadata": {
+ "id": "7KrybL96IuNW"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now we have all of the things we need. The vectorized distance matrix $\\mathbf{c}$, the constraint matrix $\\mathbf{A}$, the vectorized and concatenated original distribution $\\mathbf{b}$. We can run the linear programming optimization."
+ ],
+ "metadata": {
+ "id": "zEuEtU33S8Ly"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# We don't need the constraint that p>0 as this is the default\n",
+ "opt = linprog(c, A_eq=A, b_eq=b)\n",
+ "print(opt)"
+ ],
+ "metadata": {
+ "id": "wCfsOVbeSmF5"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Extract the answer and display"
+ ],
+ "metadata": {
+ "id": "vpkkOOI2agyl"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "TP = np.array(opt.x).reshape(10,10)\n",
+ "draw_2D_heatmap(TP,'Transport plan $\\mathbf{P}$', my_colormap)"
+ ],
+ "metadata": {
+ "id": "nZGfkrbRV_D0"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Compute the Wasserstein distance\n"
+ ],
+ "metadata": {
+ "id": "ZEiRYRVgalsJ"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "was = np.sum(TP * dist_mat)\n",
+ "print(\"Wasserstein distance = \", was)"
+ ],
+ "metadata": {
+ "id": "yiQ_8j-Raq3c"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "TODO -- Compute the\n",
+ "\n",
+ "* Forward KL divergence $D_{KL}[p,q]$ between these distributions\n",
+ "* Reverse KL divergence $D_{KL}[q,p]$ between these distributions\n",
+ "* Jensen-Shannon divergence $D_{JS}[p,q]$ between these distributions\n",
+ "\n",
+ "What do you conclude?"
+ ],
+ "metadata": {
+ "id": "zf8yTusua71s"
+ }
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap16/16_1_1D_Normalizing_Flows.ipynb b/Notebooks/Chap16/16_1_1D_Normalizing_Flows.ipynb
new file mode 100644
index 0000000..f0e7fe2
--- /dev/null
+++ b/Notebooks/Chap16/16_1_1D_Normalizing_Flows.ipynb
@@ -0,0 +1,235 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyMJLViYIpiivB2A7YIuZmzU",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 16.1: 1D normalizing flows**\n",
+ "\n",
+ "This notebook investigates a 1D normalizing flows example similar to that illustrated in figures 16.1 to 16.3 in the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ],
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "First we start with a base probability density function"
+ ],
+ "metadata": {
+ "id": "IyVn-Gi-p7wf"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the base pdf\n",
+ "def gauss_pdf(z, mu, sigma):\n",
+ " pr_z = np.exp( -0.5 * (z-mu) * (z-mu) / (sigma * sigma))/(np.sqrt(2*3.1413) * sigma)\n",
+ " return pr_z"
+ ],
+ "metadata": {
+ "id": "ZIfQwhd-AV6L"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "z = np.arange(-3,3,0.01)\n",
+ "pr_z = gauss_pdf(z, 0, 1)\n",
+ "\n",
+ "fig,ax = plt.subplots()\n",
+ "ax.plot(z, pr_z)\n",
+ "ax.set_xlim([-3,3])\n",
+ "ax.set_xlabel('$z$')\n",
+ "ax.set_ylabel('$Pr(z)$')\n",
+ "plt.show();"
+ ],
+ "metadata": {
+ "id": "gGh8RHmFp_Ls"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's define a nonlinear function that maps from the latent space $z$ to the observed data $x$."
+ ],
+ "metadata": {
+ "id": "wVXi5qIfrL9T"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define a function that maps from the base pdf over z to the observed space x\n",
+ "def f(z):\n",
+ " x1 = 6/(1+np.exp(-(z-0.25)*1.5))-3\n",
+ " x2 = z\n",
+ " p = z * z/9\n",
+ " x = (1-p) * x1 + p * x2\n",
+ " return x\n",
+ "\n",
+ "# Compute gradient of that function using finite differences\n",
+ "def df_dz(z):\n",
+ " return (f(z+0.0001)-f(z-0.0001))/0.0002"
+ ],
+ "metadata": {
+ "id": "shHdgZHjp52w"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "x = f(z)\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(z,x)\n",
+ "ax.set_xlim(-3,3)\n",
+ "ax.set_ylim(-3,3)\n",
+ "ax.set_xlabel('Latent variable, $z$')\n",
+ "ax.set_ylabel('Observed variable, $x$')\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "sz7bnCLUq3Qs"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's evaluate the density in the observed space using equation 16.1"
+ ],
+ "metadata": {
+ "id": "rmI0BbuQyXoc"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO -- plot the density in the observed space\n",
+ "# Replace these line\n",
+ "x = np.ones_like(z)\n",
+ "pr_x = np.ones_like(pr_z)\n"
+ ],
+ "metadata": {
+ "id": "iPdiT_5gyNOD"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Plot the density in the observed space\n",
+ "fig,ax = plt.subplots()\n",
+ "ax.plot(x, pr_x)\n",
+ "ax.set_xlim([-3,3])\n",
+ "ax.set_ylim([0, 0.5])\n",
+ "ax.set_xlabel('$x$')\n",
+ "ax.set_ylabel('$Pr(x)$')\n",
+ "plt.show();"
+ ],
+ "metadata": {
+ "id": "Jlks8MW3zulA"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's draw some samples from the new distribution (see section 16.1)"
+ ],
+ "metadata": {
+ "id": "1c5rO0HHz-FV"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "np.random.seed(1)\n",
+ "n_sample = 20\n",
+ "\n",
+ "# TODO -- Draw samples from the modeled density\n",
+ "# Replace this line\n",
+ "x_samples = np.ones((n_sample, 1))\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "LIlTRfpZz2k_"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Draw the samples\n",
+ "fig,ax = plt.subplots()\n",
+ "ax.plot(x, pr_x)\n",
+ "for x_sample in x_samples:\n",
+ " ax.plot([x_sample, x_sample], [0,0.1], 'r-')\n",
+ "\n",
+ "ax.set_xlim([-3,3])\n",
+ "ax.set_ylim([0, 0.5])\n",
+ "ax.set_xlabel('$x$')\n",
+ "ax.set_ylabel('$Pr(x)$')\n",
+ "plt.show();"
+ ],
+ "metadata": {
+ "id": "JS__QPNv0vUA"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap16/16_2_Autoregressive_Flows.ipynb b/Notebooks/Chap16/16_2_Autoregressive_Flows.ipynb
new file mode 100644
index 0000000..3f3b521
--- /dev/null
+++ b/Notebooks/Chap16/16_2_Autoregressive_Flows.ipynb
@@ -0,0 +1,307 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyMe8jb5kLJqkNSE/AwExTpa",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 16.2: 1D autoregressive flows**\n",
+ "\n",
+ "This notebook investigates a 1D normalizing flows example similar to that illustrated in figure 16.7 in the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ],
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "First we'll define an invertible one dimensional function as in figure 16.5"
+ ],
+ "metadata": {
+ "id": "jTK456TUd2FV"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# First let's make the 1D piecewise linear mapping as illustrated in figure 16.5\n",
+ "def g(h, phi):\n",
+ " # TODO -- write this function (equation 16.12)\n",
+ " # Note: If you have the first printing of the book, there is a mistake in equation 16.12\n",
+ " # Check the errata for the correct equation (or figure it out yourself!)\n",
+ " # Replace this line:\n",
+ " h_prime = 1\n",
+ "\n",
+ "\n",
+ " return h_prime"
+ ],
+ "metadata": {
+ "id": "zceww_9qFi00"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's test this out. If you managed to vectorize the routine above, then good for you\n",
+ "# but I'll assume you didn't and so we'll use a loop\n",
+ "\n",
+ "# Define the parameters\n",
+ "phi = np.array([0.2, 0.1, 0.4, 0.05, 0.25])\n",
+ "\n",
+ "# Run the function on an array\n",
+ "h = np.arange(0,1,0.01)\n",
+ "h_prime = np.zeros_like(h)\n",
+ "for i in range(len(h)):\n",
+ " h_prime[i] = g(h[i], phi)\n",
+ "\n",
+ "# Draw the function\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(h,h_prime, 'b-')\n",
+ "ax.set_xlim([0,1])\n",
+ "ax.set_ylim([0,1])\n",
+ "ax.set_xlabel('Input, $h$')\n",
+ "ax.set_ylabel('Output, $h^\\prime$')\n",
+ "plt.show()\n"
+ ],
+ "metadata": {
+ "id": "CLXhYl9ZIuRN"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "We will also need the inverse of this function"
+ ],
+ "metadata": {
+ "id": "zOCMYC0leOyZ"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the inverse function\n",
+ "def g_inverse(h_prime, phi):\n",
+ " # Lot's of ways to do this, but we'll just do it by bracketing\n",
+ " h_low = 0\n",
+ " h_mid = 0.5\n",
+ " h_high = 0.999\n",
+ "\n",
+ " thresh = 0.0001\n",
+ " c_iter = 0\n",
+ " while(c_iter < 20 and h_high - h_low > thresh):\n",
+ " h_prime_low = g(h_low, phi)\n",
+ " h_prime_mid = g(h_mid, phi)\n",
+ " h_prime_high = g(h_high, phi)\n",
+ " if h_prime_mid < h_prime:\n",
+ " h_low = h_mid\n",
+ " else:\n",
+ " h_high = h_mid\n",
+ "\n",
+ " h_mid = h_low+(h_high-h_low)/2\n",
+ " c_iter+=1\n",
+ "\n",
+ " return h_mid"
+ ],
+ "metadata": {
+ "id": "OIqFAgobeSM8"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's define an autoregressive flow. Let's switch to looking at figure 16.7.# We'll assume that our piecewise function will use five parameters phi1,phi2,phi3,phi4,phi5"
+ ],
+ "metadata": {
+ "id": "t8XPxipfd7hz"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "\n",
+ "def ReLU(preactivation):\n",
+ " activation = preactivation.clip(0.0)\n",
+ " return activation\n",
+ "\n",
+ "def softmax(x):\n",
+ " x = np.exp(x) ;\n",
+ " x = x/ np.sum(x) ;\n",
+ " return x\n",
+ "\n",
+ "# Return value of phi that doesn't depend on any of the inputs\n",
+ "def get_phi():\n",
+ " return np.array([0.2, 0.1, 0.4, 0.05, 0.25])\n",
+ "\n",
+ "# Compute values of phi that depend on h1\n",
+ "def shallow_network_phi_h1(h1, n_hidden=10):\n",
+ " # For neatness of code, we'll just define the parameters in the network definition itself\n",
+ " n_input = 1\n",
+ " np.random.seed(n_input)\n",
+ " beta0 = np.random.normal(size=(n_hidden,1))\n",
+ " Omega0 = np.random.normal(size=(n_hidden, n_input))\n",
+ " beta1 = np.random.normal(size=(5,1))\n",
+ " Omega1 = np.random.normal(size=(5, n_hidden))\n",
+ " return softmax(beta1 + Omega1 @ ReLU(beta0 + Omega0 @ np.array([[h1]])))\n",
+ "\n",
+ "# Compute values of phi that depend on h1 and h2\n",
+ "def shallow_network_phi_h1h2(h1,h2,n_hidden=10):\n",
+ " # For neatness of code, we'll just define the parameters in the network definition itself\n",
+ " n_input = 2\n",
+ " np.random.seed(n_input)\n",
+ " beta0 = np.random.normal(size=(n_hidden,1))\n",
+ " Omega0 = np.random.normal(size=(n_hidden, n_input))\n",
+ " beta1 = np.random.normal(size=(5,1))\n",
+ " Omega1 = np.random.normal(size=(5, n_hidden))\n",
+ " return softmax(beta1 + Omega1 @ ReLU(beta0 + Omega0 @ np.array([[h1],[h2]])))\n",
+ "\n",
+ "# Compute values of phi that depend on h1, h2, and h3\n",
+ "def shallow_network_phi_h1h2h3(h1,h2,h3, n_hidden=10):\n",
+ " # For neatness of code, we'll just define the parameters in the network definition itself\n",
+ " n_input = 3\n",
+ " np.random.seed(n_input)\n",
+ " beta0 = np.random.normal(size=(n_hidden,1))\n",
+ " Omega0 = np.random.normal(size=(n_hidden, n_input))\n",
+ " beta1 = np.random.normal(size=(5,1))\n",
+ " Omega1 = np.random.normal(size=(5, n_hidden))\n",
+ " return softmax(beta1 + Omega1 @ ReLU(beta0 + Omega0 @ np.array([[h1],[h2],[h3]])))"
+ ],
+ "metadata": {
+ "id": "PnHGlZtcNEAI"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The forward mapping as shown in figure 16.7 a"
+ ],
+ "metadata": {
+ "id": "8fXeG4V44GVH"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def forward_mapping(h1,h2,h3,h4):\n",
+ " #TODO implement the forward mapping\n",
+ " #Replace this line:\n",
+ " h_prime1 = 0 ; h_prime2=0; h_prime3=0; h_prime4 = 0\n",
+ "\n",
+ " return h_prime1, h_prime2, h_prime3, h_prime4"
+ ],
+ "metadata": {
+ "id": "N1zjnIoX0TRP"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The backward mapping as shown in figure 16.7b"
+ ],
+ "metadata": {
+ "id": "H8vQfFwI4L7r"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def backward_mapping(h1_prime,h2_prime,h3_prime,h4_prime):\n",
+ " #TODO implement the backward mapping\n",
+ " #Replace this line:\n",
+ " h1=0; h2=0; h3=0; h4 = 0\n",
+ "\n",
+ " return h1,h2,h3,h4"
+ ],
+ "metadata": {
+ "id": "HNcQTiVE4DMJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Finally, let's make sure that the network really can be inverted"
+ ],
+ "metadata": {
+ "id": "W2IxFkuyZJyn"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Test the network to see if it does invert correctly\n",
+ "h1 = 0.22; h2 = 0.41; h3 = 0.83; h4 = 0.53\n",
+ "print(\"Original h values %3.3f,%3.3f,%3.3f,%3.3f\"%(h1,h2,h3,h4))\n",
+ "h1_prime, h2_prime, h3_prime, h4_prime = forward_mapping(h1,h2,h3,h4)\n",
+ "print(\"h_prime values %3.3f,%3.3f,%3.3f,%3.3f\"%(h1_prime,h2_prime,h3_prime,h4_prime))\n",
+ "h1,h2,h3,h4 = backward_mapping(h1_prime,h2_prime,h3_prime,h4_prime)\n",
+ "print(\"Reconstructed h values %3.3f,%3.3f,%3.3f,%3.3f\"%(h1,h2,h3,h4))"
+ ],
+ "metadata": {
+ "id": "RT7qvEFp700I"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "sDknSPMLZmzh"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap16/16_3_Contraction_Mappings.ipynb b/Notebooks/Chap16/16_3_Contraction_Mappings.ipynb
new file mode 100644
index 0000000..632bfcf
--- /dev/null
+++ b/Notebooks/Chap16/16_3_Contraction_Mappings.ipynb
@@ -0,0 +1,299 @@
+{
+ "cells": [
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "colab_type": "text",
+ "id": "view-in-github"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ },
+ "source": [
+ "# **Notebook 16.3: Contraction mappings**\n",
+ "\n",
+ "This notebook investigates a 1D normalizing flows example similar to that illustrated in figure 16.9 in the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4Pfz2KSghdVI"
+ },
+ "outputs": [],
+ "source": [
+ "# Define a function that is a contraction mapping\n",
+ "def f(z):\n",
+ " return 0.3 + 0.5 *z + 0.02 * np.sin(z*15)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "zEwCbIx0hpAI"
+ },
+ "outputs": [],
+ "source": [
+ "def draw_function(f, fixed_point=None):\n",
+ " z = np.arange(0,1,0.01)\n",
+ " z_prime = f(z)\n",
+ "\n",
+ " # Draw this function\n",
+ " fig, ax = plt.subplots()\n",
+ " ax.plot(z, z_prime,'c-')\n",
+ " ax.plot([0,1],[0,1],'k--')\n",
+ " if fixed_point!=None:\n",
+ " ax.plot(fixed_point, fixed_point, 'ro')\n",
+ " ax.set_xlim(0,1)\n",
+ " ax.set_ylim(0,1)\n",
+ " ax.set_xlabel('Input, $z$')\n",
+ " ax.set_ylabel('Output, f$[z]$')\n",
+ " plt.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "k4e5Yu0fl8bz"
+ },
+ "outputs": [],
+ "source": [
+ "draw_function(f)"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "DfgKrpCAjnol"
+ },
+ "source": [
+ "Now let's find where $\\text{f}[z]=z$ using fixed point iteration"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "bAOBvZT-j3lv"
+ },
+ "outputs": [],
+ "source": [
+ "# Takes a function f and a starting point z\n",
+ "def fixed_point_iteration(f, z0):\n",
+ " # TODO -- write this function\n",
+ " # Print out the iterations as you go, so you can see the progress\n",
+ " # Set the maximum number of iterations to 20\n",
+ " # Replace this line\n",
+ " z_out = 0.5;\n",
+ "\n",
+ "\n",
+ "\n",
+ " return z_out"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "CAS0lgIomAa0"
+ },
+ "source": [
+ "Now let's test that and plot the solution"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "EYQZJdNPk8Lg"
+ },
+ "outputs": [],
+ "source": [
+ "# Now let's test that\n",
+ "z = fixed_point_iteration(f, 0.2)\n",
+ "draw_function(f, z)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4DipPiqVlnwJ"
+ },
+ "outputs": [],
+ "source": [
+ "# Let's define another function\n",
+ "def f2(z):\n",
+ " return 0.7 + -0.6 *z + 0.03 * np.sin(z*15)\n",
+ "draw_function(f2)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "tYOdbWcomdEE"
+ },
+ "outputs": [],
+ "source": [
+ "# Now let's test that\n",
+ "# TODO Before running this code, predict what you think will happen\n",
+ "z = fixed_point_iteration(f2, 0.9)\n",
+ "draw_function(f2, z)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Mni37RUpmrIu"
+ },
+ "outputs": [],
+ "source": [
+ "# Let's define another function\n",
+ "# Define a function that is a contraction mapping\n",
+ "def f3(z):\n",
+ " return -0.2 + 1.5 *z + 0.1 * np.sin(z*15)\n",
+ "draw_function(f3)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "agt5mfJrnM1O"
+ },
+ "outputs": [],
+ "source": [
+ "# Now let's test that\n",
+ "# TODO Before running this code, predict what you think will happen\n",
+ "z = fixed_point_iteration(f3, 0.7)\n",
+ "draw_function(f3, z)"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "n6GI46-ZoQz6"
+ },
+ "source": [
+ "Finally, let's invert a problem of the form $y = z+ f[z]$ for a given value of $y$. What is the $z$ that maps to it?"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "dy6r3jr9rjPf"
+ },
+ "outputs": [],
+ "source": [
+ "def f4(z):\n",
+ " return -0.3 + 0.5 *z + 0.02 * np.sin(z*15)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "GMX64Iz0nl-B"
+ },
+ "outputs": [],
+ "source": [
+ "def fixed_point_iteration_z_plus_f(f, y, z0):\n",
+ " # TODO -- write this function\n",
+ " # Replace this line\n",
+ " z_out = 1\n",
+ "\n",
+ " return z_out"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "uXxKHad5qT8Y"
+ },
+ "outputs": [],
+ "source": [
+ "def draw_function2(f, y, fixed_point=None):\n",
+ " z = np.arange(0,1,0.01)\n",
+ " z_prime = z+f(z)\n",
+ "\n",
+ " # Draw this function\n",
+ " fig, ax = plt.subplots()\n",
+ " ax.plot(z, z_prime,'c-')\n",
+ " ax.plot(z, y-f(z),'r-')\n",
+ " ax.plot([0,1],[0,1],'k--')\n",
+ " if fixed_point!=None:\n",
+ " ax.plot(fixed_point, y, 'ro')\n",
+ " ax.set_xlim(0,1)\n",
+ " ax.set_ylim(0,1)\n",
+ " ax.set_xlabel('Input, $z$')\n",
+ " ax.set_ylabel('Output, z+f$[z]$')\n",
+ " plt.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "mNEBXC3Aqd_1"
+ },
+ "outputs": [],
+ "source": [
+ "# Test this out and draw\n",
+ "y = 0.8\n",
+ "z = fixed_point_iteration_z_plus_f(f4,y,0.2)\n",
+ "draw_function2(f4,y,z)\n",
+ "# If you have done this correctly, the red dot should be\n",
+ "# where the cyan curve has a y value of 0.8"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "authorship_tag": "ABX9TyNeCWINUqqUGKMcxsqPFTAh",
+ "include_colab_link": true,
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3",
+ "name": "python3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/Notebooks/Chap17/17_1_Latent_Variable_Models.ipynb b/Notebooks/Chap17/17_1_Latent_Variable_Models.ipynb
new file mode 100644
index 0000000..b79df19
--- /dev/null
+++ b/Notebooks/Chap17/17_1_Latent_Variable_Models.ipynb
@@ -0,0 +1,405 @@
+{
+ "cells": [
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "colab_type": "text",
+ "id": "view-in-github"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ },
+ "source": [
+ "# **Notebook 17.1: Latent variable models**\n",
+ "\n",
+ "This notebook investigates a non-linear latent variable model similar to that in figures 17.2 and 17.3 of the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "import scipy\n",
+ "from matplotlib.colors import ListedColormap\n",
+ "from matplotlib import cm"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "IyVn-Gi-p7wf"
+ },
+ "source": [
+ "We'll assume that our base distribution over the latent variables is a 1D standard normal so that\n",
+ "\n",
+ "\\begin{equation}\n",
+ "Pr(z) = \\text{Norm}_{z}[0,1]\n",
+ "\\end{equation}\n",
+ "\n",
+ "As in figure 17.2, we'll assume that the output is two dimensional, we we need to define a function that maps from the 1D latent variable to two dimensions. Usually, we would use a neural network, but in this case, we'll just define an arbitrary relationship.\n",
+ "\n",
+ "\\begin{align}\n",
+ "x_{1} &=& 0.5\\cdot\\exp\\Bigl[\\sin\\bigl[2+ 3.675 z \\bigr]\\Bigr]\\\\\n",
+ "x_{2} &=& \\sin\\bigl[2+ 2.85 z \\bigr]\n",
+ "\\end{align}"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "ZIfQwhd-AV6L"
+ },
+ "outputs": [],
+ "source": [
+ "# The function that maps z to x1 and x2\n",
+ "def f(z):\n",
+ " x_1 = np.exp(np.sin(2+z*3.675)) * 0.5\n",
+ " x_2 = np.cos(2+z*2.85)\n",
+ " return x_1, x_2"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "KB9FU34onW1j"
+ },
+ "source": [
+ "Let's plot the 3D relation between the two observed variables $x_{1}$ and $x_{2}$ and the latent variables $z$ as in figure 17.2 of the book. We'll use the opacity to represent the prior probability $Pr(z)$."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "lW08xqAgnP4q"
+ },
+ "outputs": [],
+ "source": [
+ "def draw_3d_projection(z,pr_z, x1,x2):\n",
+ " alpha = pr_z / np.max(pr_z)\n",
+ " ax = plt.axes(projection='3d')\n",
+ " fig = plt.gcf()\n",
+ " fig.set_size_inches(18.5, 10.5)\n",
+ " for i in range(len(z)-1):\n",
+ " ax.plot([z[i],z[i+1]],[x1[i],x1[i+1]],[x2[i],x2[i+1]],'r-', alpha=pr_z[i])\n",
+ " ax.set_xlabel('$z$',)\n",
+ " ax.set_ylabel('$x_1$')\n",
+ " ax.set_zlabel('$x_2$')\n",
+ " ax.set_xlim(-3,3)\n",
+ " ax.set_ylim(0,2)\n",
+ " ax.set_zlim(-1,1)\n",
+ " ax.set_box_aspect((3,1,1))\n",
+ " plt.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "9DUTauMi6tPk"
+ },
+ "outputs": [],
+ "source": [
+ "# Compute the prior\n",
+ "def get_prior(z):\n",
+ " return scipy.stats.multivariate_normal.pdf(z)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "PAzHq461VqvF"
+ },
+ "outputs": [],
+ "source": [
+ "# Define the latent variable values\n",
+ "z = np.arange(-3.0,3.0,0.01)\n",
+ "# Find the probability distribution over z\n",
+ "pr_z = get_prior(z)\n",
+ "# Compute x1 and x2 for each z\n",
+ "x1,x2 = f(z)\n",
+ "# Plot the function\n",
+ "draw_3d_projection(z,pr_z, x1,x2)"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "sQg2gKR5zMrF"
+ },
+ "source": [
+ "The likelihood is defined as:\n",
+ "\\begin{align}\n",
+ " Pr(x_1,x_2|z) &=& \\text{Norm}_{[x_1,x_2]}\\Bigl[\\mathbf{f}[z],\\sigma^{2}\\mathbf{I}\\Bigr]\n",
+ "\\end{align}\n",
+ "\n",
+ "so we will also need to define the noise level $\\sigma^2$"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "In_Vg4_0nva3"
+ },
+ "outputs": [],
+ "source": [
+ "sigma_sq = 0.04"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "6P6d-AgAqxXZ"
+ },
+ "outputs": [],
+ "source": [
+ "# Draws a heatmap to represent a probability distribution, possibly with samples overlaed\n",
+ "def plot_heatmap(x1_mesh,x2_mesh,y_mesh, x1_samples=None, x2_samples=None, title=None):\n",
+ " # Define pretty colormap\n",
+ " my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ " my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ " r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ " g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ " b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ " my_colormap = ListedColormap(np.vstack((r,g,b)).transpose()/255.0)\n",
+ "\n",
+ " fig,ax = plt.subplots()\n",
+ " fig.set_size_inches(8,8)\n",
+ " ax.contourf(x1_mesh,x2_mesh,y_mesh,256,cmap=my_colormap)\n",
+ " ax.contour(x1_mesh,x2_mesh,y_mesh,8,colors=['#80808080'])\n",
+ " if title is not None:\n",
+ " ax.set_title(title);\n",
+ " if x1_samples is not None:\n",
+ " ax.plot(x1_samples, x2_samples, 'c.')\n",
+ " ax.set_xlim([-0.5,2.5])\n",
+ " ax.set_ylim([-1.5,1.5])\n",
+ " ax.set_xlabel('$x_1$'); ax.set_ylabel('$x_2$')\n",
+ " plt.show()\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "diYKb7_ZgjlJ"
+ },
+ "outputs": [],
+ "source": [
+ "# Returns the likelihood\n",
+ "def get_likelihood(x1_mesh, x2_mesh, z_val):\n",
+ " # Find the corresponding x1 and x2 values\n",
+ " x1,x2 = f(z_val)\n",
+ "\n",
+ " # Calculate the probability for a mesh of x1,x2 values.\n",
+ " mn = scipy.stats.multivariate_normal([x1, x2], [[sigma_sq, 0], [0, sigma_sq]])\n",
+ " pr_x1_x2_given_z_val = mn.pdf(np.dstack((x1_mesh, x2_mesh)))\n",
+ " return pr_x1_x2_given_z_val"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "0X4NwixzqxtZ"
+ },
+ "source": [
+ "Now let's plot the likelihood $Pr(x_1,x_2|z)$ as in fig 17.3b in the book."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "hWfqK-Oz5_DT"
+ },
+ "outputs": [],
+ "source": [
+ "# Choose some z value\n",
+ "z_val = 1.8\n",
+ "\n",
+ "# Compute the conditional distribution on a grid\n",
+ "x1_mesh, x2_mesh = np.meshgrid(np.arange(-0.5,2.5,0.01), np.arange(-1.5,1.5,0.01))\n",
+ "pr_x1_x2_given_z_val = get_likelihood(x1_mesh,x2_mesh, z_val)\n",
+ "\n",
+ "# Plot the result\n",
+ "plot_heatmap(x1_mesh, x2_mesh, pr_x1_x2_given_z_val, title=\"Conditional distribution $Pr(x_1,x_2|z)$\")\n",
+ "\n",
+ "# TODO -- Experiment with different values of z and make sure that you understand the what is happening."
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "25xqXnmFo-PH"
+ },
+ "source": [
+ "The data density is found by marginalizing over the latent variables $z$:\n",
+ "\n",
+ "\\begin{align}\n",
+ " Pr(x_1,x_2) &=& \\int Pr(x_1,x_2, z) dz \\nonumber \\\\\n",
+ " &=& \\int Pr(x_1,x_2 | z) \\cdot Pr(z)dz\\nonumber \\\\\n",
+ " &=& \\int \\text{Norm}_{[x_1,x_2]}\\Bigl[\\mathbf{f}[z],\\sigma^{2}\\mathbf{I}\\Bigr]\\cdot \\text{Norm}_{z}\\left[\\mathbf{0},\\mathbf{I}\\right]dz.\n",
+ "\\end{align}"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "H0Ijce9VzeCO"
+ },
+ "outputs": [],
+ "source": [
+ "# TODO Compute the data density\n",
+ "# We can't integrate this function in closed form\n",
+ "# So let's approximate it as a sum over the z values (z = np.arange(-3,3,0.01))\n",
+ "# You will need the functions get_likelihood() and get_prior()\n",
+ "# To make this a valid probability distribution, you need to divide\n",
+ "# By the z-increment (0.01)\n",
+ "# Replace this line\n",
+ "pr_x1_x2 = np.zeros_like(x1_mesh)\n",
+ "\n",
+ "\n",
+ "# Plot the result\n",
+ "plot_heatmap(x1_mesh, x2_mesh, pr_x1_x2, title=\"Data density $Pr(x_1,x_2)$\")\n"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "W264N7By_h9y"
+ },
+ "source": [
+ "Now let's draw some samples from the model"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Li3mK_I48k0k"
+ },
+ "outputs": [],
+ "source": [
+ "def draw_samples(n_sample):\n",
+ " # TODO Write this routine to draw n_sample samples from the model\n",
+ " # First draw a random value of z from the prior (a standard normal distribution)\n",
+ " # Then draw a sample from Pr(x1,x2|z)\n",
+ " # Replace this line\n",
+ " x1_samples=0; x2_samples = 0;\n",
+ "\n",
+ " return x1_samples, x2_samples"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "D7N7oqLe-eJO"
+ },
+ "source": [
+ "Let's plot those samples on top of the heat map."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "XRmWv99B-BWO"
+ },
+ "outputs": [],
+ "source": [
+ "x1_samples, x2_samples = draw_samples(500)\n",
+ "# Plot the result\n",
+ "plot_heatmap(x1_mesh, x2_mesh, pr_x1_x2, x1_samples, x2_samples, title=\"Data density $Pr(x_1,x_2)$\")\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "PwOjzPD5_1OF"
+ },
+ "outputs": [],
+ "source": [
+ "# Return the posterior distribution\n",
+ "def get_posterior(x1,x2):\n",
+ " z = np.arange(-3,3, 0.01)\n",
+ " # TODO -- write this function\n",
+ " # Again, we can't integrate, but we can sum\n",
+ " # We don't know the constant in the denominator of equation 17.19, but we can just normalize\n",
+ " # by the sum of the numerators for all values of z\n",
+ " # Replace this line:\n",
+ " pr_z_given_x1_x2 = np.ones_like(z)\n",
+ "\n",
+ "\n",
+ " return z, pr_z_given_x1_x2"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "PKFUY42K-Tp7"
+ },
+ "outputs": [],
+ "source": [
+ "x1 = 0.9; x2 = -0.9\n",
+ "z, pr_z_given_x1_x2 = get_posterior(x1,x2)\n",
+ "\n",
+ "\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(z, pr_z_given_x1_x2, 'r-')\n",
+ "ax.set_xlabel(\"Latent variable $z$\")\n",
+ "ax.set_ylabel(\"Posterior probability $Pr(z|x_{1},x_{2})$\")\n",
+ "ax.set_xlim([-3,3])\n",
+ "ax.set_ylim([0,1.5 * np.max(pr_z_given_x1_x2)])\n",
+ "plt.show()"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "authorship_tag": "ABX9TyOSEQVqxE5KrXmsZVh9M3gq",
+ "include_colab_link": true,
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3",
+ "name": "python3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/Notebooks/Chap17/17_2_Reparameterization_Trick.ipynb b/Notebooks/Chap17/17_2_Reparameterization_Trick.ipynb
new file mode 100644
index 0000000..588d5c4
--- /dev/null
+++ b/Notebooks/Chap17/17_2_Reparameterization_Trick.ipynb
@@ -0,0 +1,432 @@
+{
+ "cells": [
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "colab_type": "text",
+ "id": "view-in-github"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ },
+ "source": [
+ "# **Notebook 17.2: Reparameterization trick**\n",
+ "\n",
+ "This notebook investigates the reparameterization trick as described in section 17.7 of the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "paLz5RukZP1J"
+ },
+ "source": [
+ "The reparameterization trick computes the derivative of an expectation of a function $\\text{f}[x]$:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\frac{\\partial}{\\partial \\boldsymbol\\phi} \\mathbb{E}_{Pr(x|\\boldsymbol\\phi)}\\bigl[\\text{f}[x]\\bigr],\n",
+ "\\end{equation}\n",
+ "\n",
+ "with respect to the parameters $\\boldsymbol\\phi$ of the distribution $Pr(x|\\boldsymbol\\phi)$ that the expectation is over.\n",
+ "\n",
+ "Let's consider a simple concrete example, where:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "Pr(x|\\phi) = \\text{Norm}_{x}\\Bigl[\\mu, \\sigma^2\\Bigr]=\\text{Norm}_{x}\\Bigl[\\phi^3,(\\exp[\\phi])^2\\Bigr]\n",
+ "\\end{equation}\n",
+ "\n",
+ "and\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\text{f}[x] = x^2+\\sin[x]\n",
+ "\\end{equation}"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "FdEbMnDBY0i9"
+ },
+ "outputs": [],
+ "source": [
+ "# Let's approximate this expectation for a particular value of phi\n",
+ "def compute_expectation(phi, n_samples):\n",
+ " # TODO complete this function\n",
+ " # 1. Compute the mean of the normal distribution, mu\n",
+ " # 2. Compute the standard deviation of the normal distribution, sigma\n",
+ " # 3. Draw n_samples samples using np.random.normal(mu, sigma, size=(n_samples, 1))\n",
+ " # 4. Compute f[x] for each of these samples\n",
+ " # 4. Approximate the expectation by taking the average of the values of f[x]\n",
+ " # Replace this line\n",
+ " expected_f_given_phi = 0\n",
+ "\n",
+ "\n",
+ " return expected_f_given_phi"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "FTh7LJ0llNJZ"
+ },
+ "outputs": [],
+ "source": [
+ "# Set the seed so the random numbers are all the same\n",
+ "np.random.seed(0)\n",
+ "\n",
+ "# Compute the expectation for two values of phi\n",
+ "phi1 = 0.5\n",
+ "n_samples = 10000000\n",
+ "expected_f_given_phi1 = compute_expectation(phi1, n_samples)\n",
+ "print(\"Your value: \", expected_f_given_phi1, \", True value: 2.7650801613563116\")\n",
+ "\n",
+ "phi2 = -0.1\n",
+ "n_samples = 10000000\n",
+ "expected_f_given_phi2 = compute_expectation(phi2, n_samples)\n",
+ "print(\"Your value: \", expected_f_given_phi2, \", True value: 0.8176793102849222\")"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "r5Hl2QkimWx9"
+ },
+ "source": [
+ "Le't plot this expectation as a function of phi"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "05XxVLJxmkER"
+ },
+ "outputs": [],
+ "source": [
+ "phi_vals = np.arange(-1.5,1.5, 0.05)\n",
+ "expected_vals = np.zeros_like(phi_vals)\n",
+ "n_samples = 1000000\n",
+ "for i in range(len(phi_vals)):\n",
+ " expected_vals[i] = compute_expectation(phi_vals[i], n_samples)\n",
+ "\n",
+ "fig,ax = plt.subplots()\n",
+ "ax.plot(phi_vals, expected_vals,'r-')\n",
+ "ax.set_xlabel('Parameter $\\phi$')\n",
+ "ax.set_ylabel('$\\mathbb{E}_{Pr(x|\\phi)}[f[x]]$')\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "zTCykVeWqj_O"
+ },
+ "source": [
+ "It's this curve that we want to find the derivative of (so for example, we could run gradient descent and find the minimum.\n",
+ "\n",
+ "This is tricky though -- if you look at the computation that you performed, then there is a sampling step in the procedure (step 3). How do we compute the derivative of this?\n",
+ "\n",
+ "The answer is the reparameterization trick. We note that:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\text{Norm}_{x}\\Bigl[\\mu, \\sigma^2\\Bigr]=\\text{Norm}_{x}\\Bigl[0, 1\\Bigr] \\times \\sigma + \\mu\n",
+ "\\end{equation}\n",
+ "\n",
+ "and so:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\text{Norm}_{x}\\Bigl[\\phi^3,(\\exp[\\phi])^2\\Bigr] = \\text{Norm}_{x}\\Bigl[0, 1\\Bigr] \\times \\exp[\\phi]+ \\phi^3\n",
+ "\\end{equation}\n",
+ "\n",
+ "So, if we draw a sample $\\epsilon^*$ from $\\text{Norm}_{\\epsilon}[0, 1]$, then we can compute a sample $x^*$ as:\n",
+ "\n",
+ "\\begin{align}\n",
+ "x^* &=& \\epsilon^* \\times \\sigma + \\mu \\\\\n",
+ "&=& \\epsilon^* \\times \\exp[\\phi]+ \\phi^3\n",
+ "\\end{align}"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "w13HVpi9q8nF"
+ },
+ "outputs": [],
+ "source": [
+ "def compute_df_dx_star(x_star):\n",
+ " # TODO Compute this derivative (function defined at the top)\n",
+ " # Replace this line:\n",
+ " deriv = 0;\n",
+ "\n",
+ "\n",
+ "\n",
+ " return deriv\n",
+ "\n",
+ "def compute_dx_star_dphi(epsilon_star, phi):\n",
+ " # TODO Compute this derivative\n",
+ " # Replace this line:\n",
+ " deriv = 0;\n",
+ "\n",
+ "\n",
+ "\n",
+ " return deriv\n",
+ "\n",
+ "def compute_derivative_of_expectation(phi, n_samples):\n",
+ " # Generate the random values of epsilon\n",
+ " epsilon_star= np.random.normal(size=(n_samples,1))\n",
+ " # TODO -- write\n",
+ " # 1. Compute dx*/dphi using the function defined above\n",
+ " # 2. Compute x*\n",
+ " # 3. Compute df/dx* using the function you wrote above\n",
+ " # 4. Compute df/dphi = df/x* * dx*dphi\n",
+ " # 5. Average the samples of df/dphi to get the expectation.\n",
+ " # Replace this line:\n",
+ " df_dphi = 0\n",
+ "\n",
+ "\n",
+ "\n",
+ " return df_dphi"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "ntQT4An79kAl"
+ },
+ "outputs": [],
+ "source": [
+ "# Set the seed so the random numbers are all the same\n",
+ "np.random.seed(0)\n",
+ "\n",
+ "# Compute the expectation for two values of phi\n",
+ "phi1 = 0.5\n",
+ "n_samples = 10000000\n",
+ "\n",
+ "deriv = compute_derivative_of_expectation(phi1, n_samples)\n",
+ "print(\"Your value: \", deriv, \", True value: 5.726338035051403\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "t0Jqd_IN_lMU"
+ },
+ "outputs": [],
+ "source": [
+ "phi_vals = np.arange(-1.5,1.5, 0.05)\n",
+ "deriv_vals = np.zeros_like(phi_vals)\n",
+ "n_samples = 1000000\n",
+ "for i in range(len(phi_vals)):\n",
+ " deriv_vals[i] = compute_derivative_of_expectation(phi_vals[i], n_samples)\n",
+ "\n",
+ "fig,ax = plt.subplots()\n",
+ "ax.plot(phi_vals, deriv_vals,'r-')\n",
+ "ax.set_xlabel('Parameter $\\phi$')\n",
+ "ax.set_ylabel('$\\partial/\\partial\\phi\\mathbb{E}_{Pr(x|\\phi)}[f[x]]$')\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ASu4yKSwAEYI"
+ },
+ "source": [
+ "This should look plausibly like the derivative of the function we plotted above!"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "xoFR1wifc8-b"
+ },
+ "source": [
+ "The reparameterization trick computes the derivative of an expectation of a function $\\text{f}[x]$:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\frac{\\partial}{\\partial \\boldsymbol\\phi} \\mathbb{E}_{Pr(x|\\boldsymbol\\phi)}\\bigl[\\text{f}[x]\\bigr],\n",
+ "\\end{equation}\n",
+ "\n",
+ "with respect to the parameters $\\boldsymbol\\phi$ of the distribution $Pr(x|\\boldsymbol\\phi)$ that the expectation is over. This derivative can also be computed as:\n",
+ "\n",
+ "\\begin{align}\n",
+ "\\frac{\\partial}{\\partial \\boldsymbol\\phi} \\mathbb{E}_{Pr(x|\\boldsymbol\\phi)}\\bigl[\\text{f}[x]\\bigr] &=& \\mathbb{E}_{Pr(x|\\boldsymbol\\phi)}\\left[\\text{f}[x]\\frac{\\partial}{\\partial \\boldsymbol\\phi} \\log\\bigl[ Pr(x|\\boldsymbol\\phi)\\bigr]\\right]\\nonumber \\\\\n",
+ "&\\approx & \\frac{1}{I}\\sum_{i=1}^{I}\\text{f}[x_i]\\frac{\\partial}{\\partial \\boldsymbol\\phi} \\log\\bigl[ Pr(x_i|\\boldsymbol\\phi)\\bigr].\n",
+ "\\end{align}\n",
+ "\n",
+ "This method is known as the REINFORCE algorithm or score function estimator. Problem 17.5 asks you to prove this relation. Let's use this method to compute the gradient and compare.\n",
+ "\n",
+ "Recall that the expression for a univariate Gaussian is:\n",
+ "\n",
+ "\\begin{equation}\n",
+ " Pr(x|\\mu,\\sigma^2) = \\frac{1}{\\sqrt{2\\pi\\sigma^{2}}}\\exp\\left[-\\frac{(x-\\mu)^{2}}{2\\sigma^{2}}\\right].\n",
+ "\\end{equation}\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4TUaxiWvASla"
+ },
+ "outputs": [],
+ "source": [
+ "def d_log_pr_x_given_phi(x,phi):\n",
+ " # TODO -- fill in this function\n",
+ " # Compute the derivative of log[Pr(x|phi)]\n",
+ " # Replace this line:\n",
+ " deriv =0;\n",
+ "\n",
+ "\n",
+ " return deriv\n",
+ "\n",
+ "\n",
+ "def compute_derivative_of_expectation_score_function(phi, n_samples):\n",
+ " # TODO -- Compute this function\n",
+ " # 1. Calculate mu from phi\n",
+ " # 2. Calculate sigma from phi\n",
+ " # 3. Generate n_sample random samples of x using np.random.normal\n",
+ " # 4. Calculate f[x] for all of the samples\n",
+ " # 5. Multiply f[x] by d_log_pr_x_given_phi\n",
+ " # 6. Take the average of the samples\n",
+ " # Replace this line:\n",
+ " deriv = 0\n",
+ "\n",
+ "\n",
+ "\n",
+ " return deriv"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "0RSN32Rna_C_"
+ },
+ "outputs": [],
+ "source": [
+ "# Set the seed so the random numbers are all the same\n",
+ "np.random.seed(0)\n",
+ "\n",
+ "# Compute the expectation for two values of phi\n",
+ "phi1 = 0.5\n",
+ "n_samples = 100000000\n",
+ "\n",
+ "deriv = compute_derivative_of_expectation_score_function(phi1, n_samples)\n",
+ "print(\"Your value: \", deriv, \", True value: 5.724609927313369\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "EM_i5zoyElHR"
+ },
+ "outputs": [],
+ "source": [
+ "phi_vals = np.arange(-1.5,1.5, 0.05)\n",
+ "deriv_vals = np.zeros_like(phi_vals)\n",
+ "n_samples = 1000000\n",
+ "for i in range(len(phi_vals)):\n",
+ " deriv_vals[i] = compute_derivative_of_expectation_score_function(phi_vals[i], n_samples)\n",
+ "\n",
+ "fig,ax = plt.subplots()\n",
+ "ax.plot(phi_vals, deriv_vals,'r-')\n",
+ "ax.set_xlabel('Parameter $\\phi$')\n",
+ "ax.set_ylabel('$\\partial/\\partial\\phi\\mathbb{E}_{Pr(x|\\phi)}[f[x]]$')\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "1TWBiUC7bQSw"
+ },
+ "source": [
+ "This should look the same as the derivative that we computed with the reparameterization trick. So, is there any advantage to one way or the other? Let's compare the variances of the estimates\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "vV_Jx5bCbQGs"
+ },
+ "outputs": [],
+ "source": [
+ "n_estimate = 100\n",
+ "n_sample = 1000\n",
+ "phi = 0.3\n",
+ "reparam_estimates = np.zeros((n_estimate,1))\n",
+ "score_function_estimates = np.zeros((n_estimate,1))\n",
+ "for i in range(n_estimate):\n",
+ " reparam_estimates[i]= compute_derivative_of_expectation(phi, n_samples)\n",
+ " score_function_estimates[i] = compute_derivative_of_expectation_score_function(phi, n_samples)\n",
+ "\n",
+ "print(\"Variance of reparameterization estimator\", np.var(reparam_estimates))\n",
+ "print(\"Variance of score function estimator\", np.var(score_function_estimates))"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "d-0tntSYdKPR"
+ },
+ "source": [
+ "The variance of the reparameterization estimator should be quite a bit lower than the score function estimator which is why it is preferred in this situation."
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "authorship_tag": "ABX9TyOxO2/0DTH4n4zhC97qbagY",
+ "include_colab_link": true,
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3",
+ "name": "python3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/Notebooks/Chap17/17_3_Importance_Sampling.ipynb b/Notebooks/Chap17/17_3_Importance_Sampling.ipynb
new file mode 100644
index 0000000..8f31414
--- /dev/null
+++ b/Notebooks/Chap17/17_3_Importance_Sampling.ipynb
@@ -0,0 +1,507 @@
+{
+ "cells": [
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "colab_type": "text",
+ "id": "view-in-github"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ },
+ "source": [
+ "# **Notebook 17.3: Importance sampling**\n",
+ "\n",
+ "This notebook investigates importance sampling as described in section 17.8.1 of the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "f7a6xqKjkmvT"
+ },
+ "source": [
+ "Let's approximate the expectation\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\mathbb{E}_{y}\\Bigl[\\exp\\bigl[- (y-1)^4\\bigr]\\Bigr] = \\int \\exp\\bigl[- (y-1)^4\\bigr] Pr(y) dy,\n",
+ "\\end{equation}\n",
+ "\n",
+ "where\n",
+ "\n",
+ "\\begin{equation}\n",
+ "Pr(y)=\\text{Norm}_y[0,1]\n",
+ "\\end{equation}\n",
+ "\n",
+ "by drawing $I$ samples $y_i$ and using the formula:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\mathbb{E}_{y}\\Bigl[\\exp\\bigl[- (y-1)^4\\bigr]\\Bigr] \\approx \\frac{1}{I} \\sum_{i=1}^I \\exp\\bigl[-(y-1)^4 \\bigr]\n",
+ "\\end{equation}"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "VjkzRr8o2ksg"
+ },
+ "outputs": [],
+ "source": [
+ "def f(y):\n",
+ " return np.exp(-(y-1) *(y-1) *(y-1) * (y-1))\n",
+ "\n",
+ "\n",
+ "def pr_y(y):\n",
+ " return (1/np.sqrt(2*np.pi)) * np.exp(-0.5 * y * y)\n",
+ "\n",
+ "fig,ax = plt.subplots()\n",
+ "y = np.arange(-10,10,0.01)\n",
+ "ax.plot(y, f(y), 'r-', label='f$[y]$');\n",
+ "ax.plot(y, pr_y(y),'b-',label='$Pr(y)$')\n",
+ "ax.set_xlabel(\"$y$\")\n",
+ "ax.legend()\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "LGAKHjUJnWmy"
+ },
+ "outputs": [],
+ "source": [
+ "def compute_expectation(n_samples):\n",
+ " # TODO -- compute this expectation\n",
+ " # 1. Generate samples y_i using np.random.normal\n",
+ " # 2. Approximate the expectation of f[y]\n",
+ " # Replace this line\n",
+ " expectation = 0\n",
+ "\n",
+ "\n",
+ " return expectation"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "nmvixMqgodIP"
+ },
+ "outputs": [],
+ "source": [
+ "# Set the seed so the random numbers are all the same\n",
+ "np.random.seed(0)\n",
+ "\n",
+ "# Compute the expectation with a very large number of samples (good estimate)\n",
+ "n_samples = 100000000\n",
+ "expected_f= compute_expectation(n_samples)\n",
+ "print(\"Your value: \", expected_f, \", True value: 0.43160702267383166\")"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Jr4UPcqmnXCS"
+ },
+ "source": [
+ "Let's investigate how the variance of this approximation decreases as we increase the number of samples $N$.\n",
+ "\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "yrDp1ILUo08j"
+ },
+ "outputs": [],
+ "source": [
+ "def compute_mean_variance(n_sample):\n",
+ " n_estimate = 10000\n",
+ " estimates = np.zeros((n_estimate,1))\n",
+ " for i in range(n_estimate):\n",
+ " estimates[i] = compute_expectation(n_sample.astype(int))\n",
+ " return np.mean(estimates), np.var(estimates)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "BcUVsodtqdey"
+ },
+ "outputs": [],
+ "source": [
+ "# Compute the mean and variance for 1,2,... 20 samples\n",
+ "n_sample_all = np.array([1.,2,3,4,5,6,7,8,9,10,15,20,25,30,45,50,60,70,80,90,100,150,200,250,300,350,400,450,500])\n",
+ "mean_all = np.zeros_like(n_sample_all)\n",
+ "variance_all = np.zeros_like(n_sample_all)\n",
+ "for i in range(len(n_sample_all)):\n",
+ " print(\"Computing mean and variance for expectation with %d samples\"%(n_sample_all[i]))\n",
+ " mean_all[i],variance_all[i] = compute_mean_variance(n_sample_all[i])"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "feXmyk0krpUi"
+ },
+ "outputs": [],
+ "source": [
+ "fig,ax = plt.subplots()\n",
+ "ax.semilogx(n_sample_all, mean_all,'r-',label='mean estimate')\n",
+ "ax.fill_between(n_sample_all, mean_all-2*np.sqrt(variance_all), mean_all+2*np.sqrt(variance_all))\n",
+ "ax.set_xlabel(\"Number of samples\")\n",
+ "ax.set_ylabel(\"Mean of estimate\")\n",
+ "ax.plot([0,500],[0.43160702267383166,0.43160702267383166],'k--',label='true value')\n",
+ "ax.legend()\n",
+ "plt.show()\n"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "XTUpxFlSuOl7"
+ },
+ "source": [
+ "As you might expect, the more samples that we use to compute the approximate estimate, the lower the variance of the estimate."
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "6hxsl3Pxo1TT"
+ },
+ "source": [
+ " Now consider the function\n",
+ " \\begin{equation}\n",
+ " \\mbox{f}[y]= 20.446\\exp\\left[-(y-3)^4\\right],\n",
+ " \\end{equation}\n",
+ "\n",
+ "which decreases rapidly as we move away from the position $y=3$."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "znydVPW7sL4P"
+ },
+ "outputs": [],
+ "source": [
+ "def f2(y):\n",
+ " return 20.446*np.exp(- (y-3) *(y-3) *(y-3) * (y-3))\n",
+ "\n",
+ "fig,ax = plt.subplots()\n",
+ "y = np.arange(-10,10,0.01)\n",
+ "ax.plot(y, f2(y), 'r-', label='f$[y]$');\n",
+ "ax.plot(y, pr_y(y),'b-',label='$Pr(y)$')\n",
+ "ax.set_xlabel(\"$y$\")\n",
+ "ax.legend()\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "G9Xxo0OJsIqD"
+ },
+ "source": [
+ "Let's again, compute the expectation:\n",
+ "\n",
+ "\\begin{align}\n",
+ "\\mathbb{E}_{y}\\left[\\text{f}[y]\\right] &=& \\int \\text{f}[y] Pr(y) dy\\\\\n",
+ "&\\approx& \\frac{1}{I} \\text{f}[y]\n",
+ "\\end{align}\n",
+ "\n",
+ "where $Pr(y)=\\text{Norm}_y[0,1]$ by approximating with samples $y_{i}$.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "l8ZtmkA2vH4y"
+ },
+ "outputs": [],
+ "source": [
+ "def compute_expectation2(n_samples):\n",
+ " y = np.random.normal(size=(n_samples,1))\n",
+ " expectation = np.mean(f2(y))\n",
+ "\n",
+ " return expectation"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "dfUQyJ-svZ6F"
+ },
+ "outputs": [],
+ "source": [
+ "# Set the seed so the random numbers are all the same\n",
+ "np.random.seed(0)\n",
+ "\n",
+ "# Compute the expectation with a very large number of samples (good estimate)\n",
+ "n_samples = 100000000\n",
+ "expected_f2= compute_expectation2(n_samples)\n",
+ "print(\"Expected value: \", expected_f2)"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "2sVDqP0BvxqM"
+ },
+ "source": [
+ "I deliberately chose this function, because it's expectation is roughly the same as for the previous function.\n",
+ "\n",
+ "Again, let's look at the mean and the variance of the estimates"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "mHnILRkOv0Ir"
+ },
+ "outputs": [],
+ "source": [
+ "def compute_mean_variance2(n_sample):\n",
+ " n_estimate = 10000\n",
+ " estimates = np.zeros((n_estimate,1))\n",
+ " for i in range(n_estimate):\n",
+ " estimates[i] = compute_expectation2(n_sample.astype(int))\n",
+ " return np.mean(estimates), np.var(estimates)\n",
+ "\n",
+ "# Compute the variance for 1,2,... 20 samples\n",
+ "mean_all2 = np.zeros_like(n_sample_all)\n",
+ "variance_all2 = np.zeros_like(n_sample_all)\n",
+ "for i in range(len(n_sample_all)):\n",
+ " print(\"Computing variance for expectation with %d samples\"%(n_sample_all[i]))\n",
+ " mean_all2[i], variance_all2[i] = compute_mean_variance2(n_sample_all[i])"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "FkCX-hxxAnsw"
+ },
+ "outputs": [],
+ "source": [
+ "fig,ax1 = plt.subplots()\n",
+ "ax1.semilogx(n_sample_all, mean_all,'r-',label='mean estimate')\n",
+ "ax1.fill_between(n_sample_all, mean_all-2*np.sqrt(variance_all), mean_all+2*np.sqrt(variance_all))\n",
+ "ax1.set_xlabel(\"Number of samples\")\n",
+ "ax1.set_ylabel(\"Mean of estimate\")\n",
+ "ax1.plot([1,500],[0.43160702267383166,0.43160702267383166],'k--',label='true value')\n",
+ "ax1.set_ylim(-5,6)\n",
+ "ax1.set_title(\"First function\")\n",
+ "ax1.legend()\n",
+ "\n",
+ "fig2,ax2 = plt.subplots()\n",
+ "ax2.semilogx(n_sample_all, mean_all2,'r-',label='mean estimate')\n",
+ "ax2.fill_between(n_sample_all, mean_all2-2*np.sqrt(variance_all2), mean_all2+2*np.sqrt(variance_all2))\n",
+ "ax2.set_xlabel(\"Number of samples\")\n",
+ "ax2.set_ylabel(\"Mean of estimate\")\n",
+ "ax2.plot([0,500],[0.43160428638892556,0.43160428638892556],'k--',label='true value')\n",
+ "ax2.set_ylim(-5,6)\n",
+ "ax2.set_title(\"Second function\")\n",
+ "ax2.legend()\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "EtBP6NeLwZqz"
+ },
+ "source": [
+ "You can see that the variance of the estimate of the second function is considerably worse than the estimate of the variance of estimate of the first function\n",
+ "\n",
+ "TODO: Think about why this is."
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "_wuF-NoQu1--"
+ },
+ "source": [
+ " Now let's repeat this experiment with the second function, but this time use importance sampling with auxiliary distribution:\n",
+ "\n",
+ " \\begin{equation}\n",
+ " q(y)=\\text{Norm}_{y}[3,1]\n",
+ " \\end{equation}\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "jPm0AVYVIDnn"
+ },
+ "outputs": [],
+ "source": [
+ "def q_y(y):\n",
+ " return (1/np.sqrt(2*np.pi)) * np.exp(-0.5 * (y-3) * (y-3))\n",
+ "\n",
+ "def compute_expectation2b(n_samples):\n",
+ " # TODO -- complete this function\n",
+ " # 1. Draw n_samples from auxiliary distribution\n",
+ " # 2. Compute f2[y] for those samples\n",
+ " # 3. Scale the results by pr_y / q_y\n",
+ " # 4. Compute the mean of these weighted samples\n",
+ " # Replace this line\n",
+ " expectation = 0\n",
+ "\n",
+ " return expectation"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "No2ByVvOM2yQ"
+ },
+ "outputs": [],
+ "source": [
+ "# Set the seed so the random numbers are all the same\n",
+ "np.random.seed(0)\n",
+ "\n",
+ "# Compute the expectation with a very large number of samples (good estimate)\n",
+ "n_samples = 100000000\n",
+ "expected_f2= compute_expectation2b(n_samples)\n",
+ "print(\"Your value: \", expected_f2,\", True value: 0.43163734204459125 \")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "6v8Jc7z4M3Mk"
+ },
+ "outputs": [],
+ "source": [
+ "def compute_mean_variance2b(n_sample):\n",
+ " n_estimate = 10000\n",
+ " estimates = np.zeros((n_estimate,1))\n",
+ " for i in range(n_estimate):\n",
+ " estimates[i] = compute_expectation2b(n_sample.astype(int))\n",
+ " return np.mean(estimates), np.var(estimates)\n",
+ "\n",
+ "# Compute the variance for 1,2,... 20 samples\n",
+ "mean_all2b = np.zeros_like(n_sample_all)\n",
+ "variance_all2b = np.zeros_like(n_sample_all)\n",
+ "for i in range(len(n_sample_all)):\n",
+ " print(\"Computing variance for expectation with %d samples\"%(n_sample_all[i]))\n",
+ " mean_all2b[i], variance_all2b[i] = compute_mean_variance2b(n_sample_all[i])"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "C0beD4sNNM3L"
+ },
+ "outputs": [],
+ "source": [
+ "fig,ax1 = plt.subplots()\n",
+ "ax1.semilogx(n_sample_all, mean_all,'r-',label='mean estimate')\n",
+ "ax1.fill_between(n_sample_all, mean_all-2*np.sqrt(variance_all), mean_all+2*np.sqrt(variance_all))\n",
+ "ax1.set_xlabel(\"Number of samples\")\n",
+ "ax1.set_ylabel(\"Mean of estimate\")\n",
+ "ax1.plot([1,500],[0.43160702267383166,0.43160702267383166],'k--',label='true value')\n",
+ "ax1.set_ylim(-5,6)\n",
+ "ax1.set_title(\"First function\")\n",
+ "ax1.legend()\n",
+ "\n",
+ "fig2,ax2 = plt.subplots()\n",
+ "ax2.semilogx(n_sample_all, mean_all2,'r-',label='mean estimate')\n",
+ "ax2.fill_between(n_sample_all, mean_all2-2*np.sqrt(variance_all2), mean_all2+2*np.sqrt(variance_all2))\n",
+ "ax2.set_xlabel(\"Number of samples\")\n",
+ "ax2.set_ylabel(\"Mean of estimate\")\n",
+ "ax2.plot([0,500],[0.43160428638892556,0.43160428638892556],'k--',label='true value')\n",
+ "ax2.set_ylim(-5,6)\n",
+ "ax2.set_title(\"Second function\")\n",
+ "ax2.legend()\n",
+ "plt.show()\n",
+ "\n",
+ "fig2,ax2 = plt.subplots()\n",
+ "ax2.semilogx(n_sample_all, mean_all2b,'r-',label='mean estimate')\n",
+ "ax2.fill_between(n_sample_all, mean_all2b-2*np.sqrt(variance_all2b), mean_all2b+2*np.sqrt(variance_all2b))\n",
+ "ax2.set_xlabel(\"Number of samples\")\n",
+ "ax2.set_ylabel(\"Mean of estimate\")\n",
+ "ax2.plot([0,500],[ 0.43163734204459125, 0.43163734204459125],'k--',label='true value')\n",
+ "ax2.set_ylim(-5,6)\n",
+ "ax2.set_title(\"Second function with importance sampling\")\n",
+ "ax2.legend()\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "y8rgge9MNiOc"
+ },
+ "source": [
+ "You can see that the importance sampling technique has reduced the amount of variance for any given number of samples."
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "authorship_tag": "ABX9TyNecz9/CDOggPSmy1LjT/Dv",
+ "include_colab_link": true,
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3",
+ "name": "python3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/Notebooks/Chap18/18_1_Diffusion_Encoder.ipynb b/Notebooks/Chap18/18_1_Diffusion_Encoder.ipynb
new file mode 100644
index 0000000..4824fcf
--- /dev/null
+++ b/Notebooks/Chap18/18_1_Diffusion_Encoder.ipynb
@@ -0,0 +1,470 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ },
+ "source": [
+ "# **Notebook 18.1: Diffusion Encoder**\n",
+ "\n",
+ "This notebook investigates the diffusion encoder as described in section 18.2 of the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from matplotlib.colors import ListedColormap\n",
+ "from operator import itemgetter"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4PM8bf6lO0VE"
+ },
+ "outputs": [],
+ "source": [
+ "#Create pretty colormap as in book\n",
+ "my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ "my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ "r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ "g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ "b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ "my_colormap_vals = np.vstack((r,g,b)).transpose()/255.0\n",
+ "my_colormap = ListedColormap(my_colormap_vals)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "ONGRaQscfIOo"
+ },
+ "outputs": [],
+ "source": [
+ "# Probability distribution for normal\n",
+ "def norm_pdf(x, mu, sigma):\n",
+ " return np.exp(-0.5 * (x-mu) * (x-mu) / (sigma * sigma)) / np.sqrt(2*np.pi*sigma*sigma)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "gZvG0MKhfY8Y"
+ },
+ "outputs": [],
+ "source": [
+ "# True distribution is a mixture of four Gaussians\n",
+ "class TrueDataDistribution:\n",
+ " # Constructor initializes parameters\n",
+ " def __init__(self):\n",
+ " self.mu = [1.5, -0.216, 0.45, -1.875]\n",
+ " self.sigma = [0.3, 0.15, 0.525, 0.075]\n",
+ " self.w = [0.2, 0.3, 0.35, 0.15]\n",
+ "\n",
+ " # Return PDF\n",
+ " def pdf(self, x):\n",
+ " return(self.w[0] *norm_pdf(x,self.mu[0],self.sigma[0]) + self.w[1] *norm_pdf(x,self.mu[1],self.sigma[1]) + self.w[2] *norm_pdf(x,self.mu[2],self.sigma[2]) + self.w[3] *norm_pdf(x,self.mu[3],self.sigma[3]))\n",
+ "\n",
+ " # Draw samples\n",
+ " def sample(self, n):\n",
+ " hidden = np.random.choice(4, n, p=self.w)\n",
+ " epsilon = np.random.normal(size=(n))\n",
+ " mu_list = list(itemgetter(*hidden)(self.mu))\n",
+ " sigma_list = list(itemgetter(*hidden)(self.sigma))\n",
+ " return mu_list + sigma_list * epsilon"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "qXmej3TUuQyp"
+ },
+ "outputs": [],
+ "source": [
+ "# Define ground truth probability distribution that we will model\n",
+ "true_dist = TrueDataDistribution()\n",
+ "# Let's visualize this\n",
+ "x_vals = np.arange(-3,3,0.01)\n",
+ "pr_x_true = true_dist.pdf(x_vals)\n",
+ "fig,ax = plt.subplots()\n",
+ "ax.plot(x_vals, pr_x_true, 'r-')\n",
+ "ax.set_xlabel(\"$x$\")\n",
+ "ax.set_ylabel(\"$Pr(x)$\")\n",
+ "ax.set_ylim(0,1.0)\n",
+ "ax.set_xlim(-3,3)\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "XHdtfRP47YLy"
+ },
+ "source": [
+ "Let's first implement the forward process"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "hkApJ2VJlQuk"
+ },
+ "outputs": [],
+ "source": [
+ "# Do one step of diffusion (equation 18.1)\n",
+ "def diffuse_one_step(z_t_minus_1, beta_t):\n",
+ " # TODO -- Implement this function\n",
+ " # Use np.random.normal to generate the samples epsilon\n",
+ " # Replace this line\n",
+ " z_t = np.zeros_like(z_t_minus_1)\n",
+ "\n",
+ " return z_t"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ECAUfHNi9NVW"
+ },
+ "source": [
+ "Now let's run the diffusion process for a whole bunch of samples"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "M-TY5w9Q8LYW"
+ },
+ "outputs": [],
+ "source": [
+ "# Generate some samples\n",
+ "n_sample = 10000\n",
+ "np.random.seed(6)\n",
+ "x = true_dist.sample(n_sample)\n",
+ "\n",
+ "# Number of time steps\n",
+ "T = 100\n",
+ "# Noise schedule has same value at every time step\n",
+ "beta = 0.01511\n",
+ "\n",
+ "# We'll store the diffused samples in an array\n",
+ "samples = np.zeros((T+1, n_sample))\n",
+ "samples[0,:] = x\n",
+ "\n",
+ "for t in range(T):\n",
+ " samples[t+1,:] = diffuse_one_step(samples[t,:], beta)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "jYrAW6tN-gJ4"
+ },
+ "source": [
+ "Let's, plot the evolution of a few paths as in figure 18.2"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4XU6CDZC_kFo"
+ },
+ "outputs": [],
+ "source": [
+ "fig, ax = plt.subplots()\n",
+ "t_vals = np.arange(0,101,1)\n",
+ "ax.plot(samples[:,0],t_vals,'r-')\n",
+ "ax.plot(samples[:,1],t_vals,'g-')\n",
+ "ax.plot(samples[:,2],t_vals,'b-')\n",
+ "ax.plot(samples[:,3],t_vals,'c-')\n",
+ "ax.plot(samples[:,4],t_vals,'m-')\n",
+ "ax.set_xlim([-3,3])\n",
+ "ax.set_ylim([101, 0])\n",
+ "ax.set_xlabel('value')\n",
+ "ax.set_ylabel('z_{t}')\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "SGTYGGevAktz"
+ },
+ "source": [
+ "Notice that the samples have a tendency to move toward the center. Now let's look at the histogram of the samples at each stage"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "bn5E5NzL-evM"
+ },
+ "outputs": [],
+ "source": [
+ "def draw_hist(z_t,title=''):\n",
+ " fig, ax = plt.subplots()\n",
+ " fig.set_size_inches(8,2.5)\n",
+ " plt.hist(z_t , bins=np.arange(-3,3, 0.1), density = True)\n",
+ " ax.set_xlim([-3,3])\n",
+ " ax.set_ylim([0,1.0])\n",
+ " ax.set_title(title)\n",
+ " plt.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "pn_XD-EhBlwk"
+ },
+ "outputs": [],
+ "source": [
+ "draw_hist(samples[0,:],'Original data')\n",
+ "draw_hist(samples[5,:],'Time step 5')\n",
+ "draw_hist(samples[10,:],'Time step 10')\n",
+ "draw_hist(samples[20,:],'Time step 20')\n",
+ "draw_hist(samples[40,:],'Time step 40')\n",
+ "draw_hist(samples[80,:],'Time step 80')\n",
+ "draw_hist(samples[100,:],'Time step 100')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "skuLfGl5Czf4"
+ },
+ "source": [
+ "You can clearly see that as the diffusion process continues, the data becomes more Gaussian."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "s37CBSzzK7wh"
+ },
+ "source": [
+ "Now let's investigate the diffusion kernel as in figure 18.3 of the book.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "vL62Iym0LEtY"
+ },
+ "outputs": [],
+ "source": [
+ "def diffusion_kernel(x, t, beta):\n",
+ " # TODO -- write this function\n",
+ " # Replace this line:\n",
+ " dk_mean = 0.0 ; dk_std = 1.0\n",
+ "\n",
+ " return dk_mean, dk_std"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "KtP1KF8wMh8o"
+ },
+ "outputs": [],
+ "source": [
+ "def draw_prob_dist(x_plot_vals, prob_dist, title=''):\n",
+ " fig, ax = plt.subplots()\n",
+ " fig.set_size_inches(8,2.5)\n",
+ " ax.plot(x_plot_vals, prob_dist, 'b-')\n",
+ " ax.set_xlim([-3,3])\n",
+ " ax.set_ylim([0,1.0])\n",
+ " ax.set_title(title)\n",
+ " plt.show()\n",
+ "\n",
+ "def compute_and_plot_diffusion_kernels(x, T, beta, my_colormap):\n",
+ " x_plot_vals = np.arange(-3,3,0.01)\n",
+ " diffusion_kernels = np.zeros((T+1,len(x_plot_vals)))\n",
+ " dk_mean_all = np.ones((T+1,1))*x\n",
+ " dk_std_all = np.zeros((T+1,1))\n",
+ " for t in range(T):\n",
+ " dk_mean_all[t+1], dk_std_all[t+1] = diffusion_kernel(x,t+1,beta)\n",
+ " diffusion_kernels[t+1,:] = norm_pdf(x_plot_vals, dk_mean_all[t+1], dk_std_all[t+1])\n",
+ "\n",
+ " samples = np.ones((T+1, 5))\n",
+ " samples[0,:] = x\n",
+ "\n",
+ " for t in range(T):\n",
+ " samples[t+1,:] = diffuse_one_step(samples[t,:], beta)\n",
+ "\n",
+ " fig, ax = plt.subplots()\n",
+ " fig.set_size_inches(6,6)\n",
+ "\n",
+ " # Plot the image containing all the kernels\n",
+ " plt.imshow(diffusion_kernels, cmap=my_colormap, interpolation='nearest')\n",
+ "\n",
+ " # Plot +/- 2 standard deviations\n",
+ " ax.plot((dk_mean_all -2 * dk_std_all)/0.01 + len(x_plot_vals)/2, np.arange(0,101,1),'y--')\n",
+ " ax.plot((dk_mean_all +2 * dk_std_all)/0.01 + len(x_plot_vals)/2, np.arange(0,101,1),'y--')\n",
+ "\n",
+ " # Plot a few trajectories\n",
+ " ax.plot(samples[:,0]/0.01 + + len(x_plot_vals)/2, np.arange(0,101,1), 'r-')\n",
+ " ax.plot(samples[:,1]/0.01 + + len(x_plot_vals)/2, np.arange(0,101,1), 'g-')\n",
+ " ax.plot(samples[:,2]/0.01 + + len(x_plot_vals)/2, np.arange(0,101,1), 'b-')\n",
+ " ax.plot(samples[:,3]/0.01 + + len(x_plot_vals)/2, np.arange(0,101,1), 'c-')\n",
+ " ax.plot(samples[:,4]/0.01 + + len(x_plot_vals)/2, np.arange(0,101,1), 'm-')\n",
+ "\n",
+ " # Tidy up and plot\n",
+ " ax.set_ylabel(\"$Pr(z_{t}|x)$\")\n",
+ " ax.get_xaxis().set_visible(False)\n",
+ " ax.set_xlim([0,601])\n",
+ " ax.set_aspect(601/T)\n",
+ " plt.show()\n",
+ "\n",
+ "\n",
+ " draw_prob_dist(x_plot_vals, diffusion_kernels[20,:],'$q(z_{20}|x)$')\n",
+ " draw_prob_dist(x_plot_vals, diffusion_kernels[40,:],'$q(z_{40}|x)$')\n",
+ " draw_prob_dist(x_plot_vals, diffusion_kernels[80,:],'$q(z_{80}|x)$')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "g8TcI5wtRQsx"
+ },
+ "outputs": [],
+ "source": [
+ "x = -2\n",
+ "compute_and_plot_diffusion_kernels(x, T, beta, my_colormap)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "-RuN2lR28-hK"
+ },
+ "source": [
+ "TODO -- Run this for different version of $x$ and check that you understand how the graphs change"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "n-x6Whz2J_zy"
+ },
+ "source": [
+ "Finally, let's estimate the marginal distributions empirically and visualize them as in figure 18.4 of the book. This is only tractable because the data is in one dimension and we know the original distribution.\n",
+ "\n",
+ "The marginal distribution at time t is the sum of the diffusion kernels for each position x, weighted by the probability of seeing that value of x in the true distribution."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "YzN5duYpg7C-"
+ },
+ "outputs": [],
+ "source": [
+ "def diffusion_marginal(x_plot_vals, pr_x_true, t, beta):\n",
+ " # If time is zero then marginal is just original distribution\n",
+ " if t == 0:\n",
+ " return pr_x_true\n",
+ "\n",
+ " # The thing we are computing\n",
+ " marginal_at_time_t = np.zeros_like(pr_x_true);\n",
+ "\n",
+ "\n",
+ " # TODO Write this function\n",
+ " # 1. For each x (value in x_plot_vals):\n",
+ " # 2. Compute the mean and variance of the diffusion kernel at time t\n",
+ " # 3. Compute pdf of this Gaussian at every x_plot_val\n",
+ " # 4. Weight Gaussian by probability at position x and by 0.01 to componensate for bin size\n",
+ " # 5. Accumulate weighted Gaussian in marginal at time t.\n",
+ "\n",
+ " # Replace this line:\n",
+ " marginal_at_time_t = marginal_at_time_t\n",
+ "\n",
+ "\n",
+ "\n",
+ " return marginal_at_time_t"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "OgEU9sxjRaeO"
+ },
+ "outputs": [],
+ "source": [
+ "x_plot_vals = np.arange(-3,3,0.01)\n",
+ "marginal_distributions = np.zeros((T+1,len(x_plot_vals)))\n",
+ "\n",
+ "for t in range(T+1):\n",
+ " marginal_distributions[t,:] = diffusion_marginal(x_plot_vals, pr_x_true , t, beta)\n",
+ "\n",
+ "fig, ax = plt.subplots()\n",
+ "fig.set_size_inches(6,6)\n",
+ "\n",
+ "# Plot the image containing all the kernels\n",
+ "plt.imshow(marginal_distributions, cmap=my_colormap, interpolation='nearest')\n",
+ "\n",
+ "# Tidy up and plot\n",
+ "ax.set_ylabel(\"$Pr(z_{t})$\")\n",
+ "ax.get_xaxis().set_visible(False)\n",
+ "ax.set_xlim([0,601])\n",
+ "ax.set_aspect(601/T)\n",
+ "plt.show()\n",
+ "\n",
+ "\n",
+ "draw_prob_dist(x_plot_vals, marginal_distributions[0,:],'$q(z_{0})$')\n",
+ "draw_prob_dist(x_plot_vals, marginal_distributions[20,:],'$q(z_{20})$')\n",
+ "draw_prob_dist(x_plot_vals, marginal_distributions[60,:],'$q(z_{60})$')"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "display_name": "Python 3",
+ "name": "python3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
\ No newline at end of file
diff --git a/Notebooks/Chap18/18_2_1D_Diffusion_Model.ipynb b/Notebooks/Chap18/18_2_1D_Diffusion_Model.ipynb
new file mode 100644
index 0000000..42b7c65
--- /dev/null
+++ b/Notebooks/Chap18/18_2_1D_Diffusion_Model.ipynb
@@ -0,0 +1,388 @@
+{
+ "cells": [
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "colab_type": "text",
+ "id": "view-in-github"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ },
+ "source": [
+ "# **Notebook 18.2: 1D Diffusion Model**\n",
+ "\n",
+ "This notebook investigates the diffusion encoder as described in section 18.3 and 18.4 of the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from matplotlib.colors import ListedColormap\n",
+ "from operator import itemgetter\n",
+ "from scipy import stats\n",
+ "from IPython.display import display, clear_output"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4PM8bf6lO0VE"
+ },
+ "outputs": [],
+ "source": [
+ "#Create pretty colormap as in book\n",
+ "my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ "my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ "r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ "g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ "b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ "my_colormap_vals = np.vstack((r,g,b)).transpose()/255.0\n",
+ "my_colormap = ListedColormap(my_colormap_vals)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "ONGRaQscfIOo"
+ },
+ "outputs": [],
+ "source": [
+ "# Probability distribution for normal\n",
+ "def norm_pdf(x, mu, sigma):\n",
+ " return np.exp(-0.5 * (x-mu) * (x-mu) / (sigma * sigma)) / np.sqrt(2*np.pi*sigma*sigma)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "gZvG0MKhfY8Y"
+ },
+ "outputs": [],
+ "source": [
+ "# True distribution is a mixture of four Gaussians\n",
+ "class TrueDataDistribution:\n",
+ " # Constructor initializes parameters\n",
+ " def __init__(self):\n",
+ " self.mu = [1.5, -0.216, 0.45, -1.875]\n",
+ " self.sigma = [0.3, 0.15, 0.525, 0.075]\n",
+ " self.w = [0.2, 0.3, 0.35, 0.15]\n",
+ "\n",
+ " # Return PDF\n",
+ " def pdf(self, x):\n",
+ " return(self.w[0] *norm_pdf(x,self.mu[0],self.sigma[0]) + self.w[1] *norm_pdf(x,self.mu[1],self.sigma[1]) + self.w[2] *norm_pdf(x,self.mu[2],self.sigma[2]) + self.w[3] *norm_pdf(x,self.mu[3],self.sigma[3]))\n",
+ "\n",
+ " # Draw samples\n",
+ " def sample(self, n):\n",
+ " hidden = np.random.choice(4, n, p=self.w)\n",
+ " epsilon = np.random.normal(size=(n))\n",
+ " mu_list = list(itemgetter(*hidden)(self.mu))\n",
+ " sigma_list = list(itemgetter(*hidden)(self.sigma))\n",
+ " return mu_list + sigma_list * epsilon"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "iJu_uBiaeUVv"
+ },
+ "outputs": [],
+ "source": [
+ "# Define ground truth probability distribution that we will model\n",
+ "true_dist = TrueDataDistribution()\n",
+ "# Let's visualize this\n",
+ "x_vals = np.arange(-3,3,0.01)\n",
+ "pr_x_true = true_dist.pdf(x_vals)\n",
+ "fig,ax = plt.subplots()\n",
+ "fig.set_size_inches(8,2.5)\n",
+ "ax.plot(x_vals, pr_x_true, 'r-')\n",
+ "ax.set_xlabel(\"$x$\")\n",
+ "ax.set_ylabel(\"$Pr(x)$\")\n",
+ "ax.set_ylim(0,1.0)\n",
+ "ax.set_xlim(-3,3)\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "DRHUG_41i4t_"
+ },
+ "source": [
+ "To train the model to describe this distribution, we'll need to generate pairs of samples drawn from $Pr(z_t|x)$ (diffusion kernel) and $q(z_{t-1}|z_{t},x)$ (equation 18.15).\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "x6B8t72Ukscd"
+ },
+ "outputs": [],
+ "source": [
+ "# The diffusion kernel returns the parameters of Pr(z_{t}|x)\n",
+ "def diffusion_kernel(x, t, beta):\n",
+ " alpha = np.power(1-beta,t)\n",
+ " dk_mean = x * np.sqrt(alpha)\n",
+ " dk_std = np.sqrt(1-alpha)\n",
+ " return dk_mean, dk_std\n",
+ "\n",
+ "# Compute mean and variance q(z_{t-1}|z_{t},x)\n",
+ "def conditional_diffusion_distribution(x,z_t,t,beta):\n",
+ " # TODO -- Implement this function\n",
+ " # Replace this line\n",
+ " cd_mean = 0; cd_std = 1\n",
+ "\n",
+ " return cd_mean, cd_std\n",
+ "\n",
+ "def get_data_pairs(x_train,t,beta):\n",
+ " # Find diffusion kernel for every x_train and draw samples\n",
+ " dk_mean, dk_std = diffusion_kernel(x_train, t, beta)\n",
+ " z_t = np.random.normal(size=x_train.shape) * dk_std + dk_mean\n",
+ " # Find conditional diffusion distribution for each x_train, z pair and draw samples\n",
+ " cd_mean, cd_std = conditional_diffusion_distribution(x_train,z_t,t,beta)\n",
+ " if t == 1:\n",
+ " z_tminus1 = x_train\n",
+ " else:\n",
+ " z_tminus1 = np.random.normal(size=x_train.shape) * cd_std + cd_mean\n",
+ "\n",
+ " return z_t, z_tminus1"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "aSG_4uA8_zZ-"
+ },
+ "source": [
+ "We also need models $\\text{f}_t[z_{t},\\phi_{t}]$ that map from $z_{t}$ to the mean of the distribution at time $z_{t-1}$. We're just going to use a very hacky non-parametric model (basically a lookup table) that tells you the result based on the (quantized) input."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "ZHViC0pL_yy5"
+ },
+ "outputs": [],
+ "source": [
+ "# This code is really ugly! Don't look too closely at it!\n",
+ "# All you need to know is that it is a model that trains from pairs zt, zt_minus1\n",
+ "# And can then predict zt\n",
+ "class NonParametricModel():\n",
+ " # Constructor initializes parameters\n",
+ " def __init__(self):\n",
+ "\n",
+ " self.inc = 0.01\n",
+ " self.max_val = 3.0\n",
+ " self.model = []\n",
+ "\n",
+ " # Learns a model that predicts z_t_minus1 given z_t\n",
+ " def train(self, zt, zt_minus1):\n",
+ " zt = np.clip(zt,-self.max_val,self.max_val)\n",
+ " zt_minus1 = np.clip(zt_minus1,-self.max_val,self.max_val)\n",
+ " bins = np.arange(-self.max_val,self.max_val+self.inc,self.inc)\n",
+ " numerator, *_ = stats.binned_statistic(zt, zt_minus1-zt, statistic='sum',bins=bins)\n",
+ " denominator, *_ = stats.binned_statistic(zt, zt_minus1-zt, statistic='count',bins=bins)\n",
+ " self.model = numerator / (denominator + 1)\n",
+ "\n",
+ " def predict(self, zt):\n",
+ " bin_index = np.floor((zt+self.max_val)/self.inc)\n",
+ " bin_index = np.clip(bin_index,0, len(self.model)-1).astype('uint32')\n",
+ " return zt + self.model[bin_index]"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "CzVFybWoBygu"
+ },
+ "outputs": [],
+ "source": [
+ "# Sample data from distribution (this would usually be our collected training set)\n",
+ "n_sample = 100000\n",
+ "x_train = true_dist.sample(n_sample)\n",
+ "\n",
+ "# Define model parameters\n",
+ "T = 100\n",
+ "beta = 0.01511\n",
+ "\n",
+ "all_models = []\n",
+ "for t in range(0,T):\n",
+ " clear_output(wait=True)\n",
+ " display(\"Training timestep %d\"%(t))\n",
+ " zt,zt_minus1 = get_data_pairs(x_train,t+1,beta)\n",
+ " all_models.append(NonParametricModel())\n",
+ " # The model at index t maps data from z_{t+1} to z_{t}\n",
+ " all_models[t].train(zt,zt_minus1)"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ZPc9SEvtl14U"
+ },
+ "source": [
+ "Now that we've learned the model, let's draw some samples from it. We start at $z_{100}$ and use the model to predict $z_{99}$, then $z_{98}$ and so on until finally we get to $z_{1}$ and then $x$ (represented as $z_{0}$ here). We'll store all of the intermediate stages as well, so we can plot the trajectories. See equations 18.16."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "A-ZMFOvACIOw"
+ },
+ "outputs": [],
+ "source": [
+ "def sample(model, T, sigma_t, n_samples):\n",
+ " # Create the output array\n",
+ " # Each row represents a time step, first row will be sampled data\n",
+ " # Each column represents a different sample\n",
+ " samples = np.zeros((T+1,n_samples))\n",
+ "\n",
+ " # TODO -- Initialize the samples z_{T} at samples[T,:] from standard normal distribution\n",
+ " # Replace this line\n",
+ " samples[T,:] = np.zeros((1,n_samples))\n",
+ "\n",
+ "\n",
+ " # For t=100...99..98... ...0\n",
+ " for t in range(T,0,-1):\n",
+ " clear_output(wait=True)\n",
+ " display(\"Predicting z_{%d} from z_{%d}\"%(t-1,t))\n",
+ " # TODO Predict samples[t-1,:] from samples[t,:] using the appropriate model\n",
+ " # Replace this line:\n",
+ " samples[t-1,:] = np.zeros((1,n_samples))\n",
+ "\n",
+ "\n",
+ " # If not the last time step\n",
+ " if t>0:\n",
+ " # TODO Add noise to the samples at z_t-1 we just generated with mean zero, standard deviation sigma_t\n",
+ " # Replace this line\n",
+ " samples[t-1,:] = samples[t-1,:]\n",
+ "\n",
+ " return samples"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ECAUfHNi9NVW"
+ },
+ "source": [
+ "Now let's run the diffusion process for a whole bunch of samples"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "M-TY5w9Q8LYW"
+ },
+ "outputs": [],
+ "source": [
+ "sigma_t=0.12288\n",
+ "n_samples = 100000\n",
+ "samples = sample(all_models, T, sigma_t, n_samples)\n",
+ "\n",
+ "\n",
+ "# Plot the data\n",
+ "sampled_data = samples[0,:]\n",
+ "bins = np.arange(-3,3.05,0.05)\n",
+ "\n",
+ "fig,ax = plt.subplots()\n",
+ "fig.set_size_inches(8,2.5)\n",
+ "ax.set_xlim([-3,3])\n",
+ "plt.hist(sampled_data, bins=bins, density =True)\n",
+ "ax.set_ylim(0, 0.8)\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "jYrAW6tN-gJ4"
+ },
+ "source": [
+ "Let's, plot the evolution of a few of the paths as in figure 18.7 (paths are from bottom to top now)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4XU6CDZC_kFo"
+ },
+ "outputs": [],
+ "source": [
+ "fig, ax = plt.subplots()\n",
+ "t_vals = np.arange(0,101,1)\n",
+ "ax.plot(samples[:,0],t_vals,'r-')\n",
+ "ax.plot(samples[:,1],t_vals,'g-')\n",
+ "ax.plot(samples[:,2],t_vals,'b-')\n",
+ "ax.plot(samples[:,3],t_vals,'c-')\n",
+ "ax.plot(samples[:,4],t_vals,'m-')\n",
+ "ax.set_xlim([-3,3])\n",
+ "ax.set_ylim([101, 0])\n",
+ "ax.set_xlabel('value')\n",
+ "ax.set_ylabel('z_{t}')\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "SGTYGGevAktz"
+ },
+ "source": [
+ "Notice that the samples have a tendency to move from positions that are near the center at time 100 to positions that are high in the true probability distribution at time 0"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "authorship_tag": "ABX9TyM4DdZDGoP1xGst+Nn+rwvt",
+ "include_colab_link": true,
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3",
+ "name": "python3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/Notebooks/Chap18/18_3_Reparameterized_Model.ipynb b/Notebooks/Chap18/18_3_Reparameterized_Model.ipynb
new file mode 100644
index 0000000..eac07b7
--- /dev/null
+++ b/Notebooks/Chap18/18_3_Reparameterized_Model.ipynb
@@ -0,0 +1,370 @@
+{
+ "cells": [
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "colab_type": "text",
+ "id": "view-in-github"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ },
+ "source": [
+ "# **Notebook 18.3: 1D Reparameterized model**\n",
+ "\n",
+ "This notebook investigates the reparameterized model as described in section 18.5 of the book and implements algorithms 18.1 and 18.2.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from matplotlib.colors import ListedColormap\n",
+ "from operator import itemgetter\n",
+ "from scipy import stats\n",
+ "from IPython.display import display, clear_output"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4PM8bf6lO0VE"
+ },
+ "outputs": [],
+ "source": [
+ "#Create pretty colormap as in book\n",
+ "my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ "my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ "r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ "g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ "b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ "my_colormap_vals = np.vstack((r,g,b)).transpose()/255.0\n",
+ "my_colormap = ListedColormap(my_colormap_vals)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "ONGRaQscfIOo"
+ },
+ "outputs": [],
+ "source": [
+ "# Probability distribution for normal\n",
+ "def norm_pdf(x, mu, sigma):\n",
+ " return np.exp(-0.5 * (x-mu) * (x-mu) / (sigma * sigma)) / np.sqrt(2*np.pi*sigma*sigma)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "gZvG0MKhfY8Y"
+ },
+ "outputs": [],
+ "source": [
+ "# True distribution is a mixture of four Gaussians\n",
+ "class TrueDataDistribution:\n",
+ " # Constructor initializes parameters\n",
+ " def __init__(self):\n",
+ " self.mu = [1.5, -0.216, 0.45, -1.875]\n",
+ " self.sigma = [0.3, 0.15, 0.525, 0.075]\n",
+ " self.w = [0.2, 0.3, 0.35, 0.15]\n",
+ "\n",
+ " # Return PDF\n",
+ " def pdf(self, x):\n",
+ " return(self.w[0] *norm_pdf(x,self.mu[0],self.sigma[0]) + self.w[1] *norm_pdf(x,self.mu[1],self.sigma[1]) + self.w[2] *norm_pdf(x,self.mu[2],self.sigma[2]) + self.w[3] *norm_pdf(x,self.mu[3],self.sigma[3]))\n",
+ "\n",
+ " # Draw samples\n",
+ " def sample(self, n):\n",
+ " hidden = np.random.choice(4, n, p=self.w)\n",
+ " epsilon = np.random.normal(size=(n))\n",
+ " mu_list = list(itemgetter(*hidden)(self.mu))\n",
+ " sigma_list = list(itemgetter(*hidden)(self.sigma))\n",
+ " return mu_list + sigma_list * epsilon"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "iJu_uBiaeUVv"
+ },
+ "outputs": [],
+ "source": [
+ "# Define ground truth probability distribution that we will model\n",
+ "true_dist = TrueDataDistribution()\n",
+ "# Let's visualize this\n",
+ "x_vals = np.arange(-3,3,0.01)\n",
+ "pr_x_true = true_dist.pdf(x_vals)\n",
+ "fig,ax = plt.subplots()\n",
+ "fig.set_size_inches(8,2.5)\n",
+ "ax.plot(x_vals, pr_x_true, 'r-')\n",
+ "ax.set_xlabel(\"$x$\")\n",
+ "ax.set_ylabel(\"$Pr(x)$\")\n",
+ "ax.set_ylim(0,1.0)\n",
+ "ax.set_xlim(-3,3)\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "DRHUG_41i4t_"
+ },
+ "source": [
+ "To train the model to describe this distribution, we'll need to generate pairs of samples drawn from $Pr(z_t|x)$ (diffusion kernel) and $q(z_{t-1}|z_{t},x)$ (equation 18.15).\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "x6B8t72Ukscd"
+ },
+ "outputs": [],
+ "source": [
+ "# Return z_t (the argument of g_{t}[] in the loss function in algorithm 18.1) and epsilon\n",
+ "def get_data_pairs(x_train,t,beta):\n",
+ " # TODO -- write this function\n",
+ " # Replace these lines\n",
+ " epsilon = np.ones_like(x_train)\n",
+ " z_t = np.ones_like(x_train)\n",
+ "\n",
+ " return z_t, epsilon"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "aSG_4uA8_zZ-"
+ },
+ "source": [
+ "We also need models $\\text{g}_t[z_{t},\\phi_{t}]$ that map from $z_{t}$ to the noise $\\epsilon$ that was added. We're just going to use a very hacky non-parametric model (basically a lookup table) that tells you the result based on the (quantized) input."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "ZHViC0pL_yy5"
+ },
+ "outputs": [],
+ "source": [
+ "# This code is really ugly! Don't look too closely at it!\n",
+ "# All you need to know is that it is a model that trains from pairs zt, zt_minus1\n",
+ "# And can then predict zt\n",
+ "class NonParametricModel():\n",
+ " # Constructor initializes parameters\n",
+ " def __init__(self):\n",
+ "\n",
+ " self.inc = 0.01\n",
+ " self.max_val = 3.0\n",
+ " self.model = []\n",
+ "\n",
+ " # Learns a model that predicts epsilon given z_t\n",
+ " def train(self, zt, epsilon):\n",
+ " zt = np.clip(zt,-self.max_val,self.max_val)\n",
+ " epsilon = np.clip(epsilon,-self.max_val,self.max_val)\n",
+ " bins = np.arange(-self.max_val,self.max_val+self.inc,self.inc)\n",
+ " numerator, *_ = stats.binned_statistic(zt, epsilon, statistic='sum',bins=bins)\n",
+ " denominator, *_ = stats.binned_statistic(zt, epsilon, statistic='count',bins=bins)\n",
+ " self.model = numerator / (denominator + 1)\n",
+ "\n",
+ " def predict(self, zt):\n",
+ " bin_index = np.floor((zt+self.max_val)/self.inc)\n",
+ " bin_index = np.clip(bin_index,0, len(self.model)-1).astype('uint32')\n",
+ " return self.model[bin_index]"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "CzVFybWoBygu"
+ },
+ "outputs": [],
+ "source": [
+ "# Sample data from distribution (this would usually be our collected training set)\n",
+ "n_sample = 100000\n",
+ "x_train = true_dist.sample(n_sample)\n",
+ "\n",
+ "# Define model parameters\n",
+ "T = 100\n",
+ "beta = 0.01511\n",
+ "\n",
+ "all_models = []\n",
+ "for t in range(0,T):\n",
+ " clear_output(wait=True)\n",
+ " display(\"Training timestep %d\"%(t))\n",
+ " zt,epsilon= get_data_pairs(x_train,t,beta)\n",
+ " all_models.append(NonParametricModel())\n",
+ " # The model at index t maps data from z_{t+1} to epsilon\n",
+ " all_models[t].train(zt,epsilon)"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ZPc9SEvtl14U"
+ },
+ "source": [
+ "Now that we've learned the model, let's draw some samples from it. We start at $z_{100}$ and use the model to predict $z_{99}$, then $z_{98}$ and so on until finally we get to $z_{1}$ and then $x$ (represented as $z_{0}$ here). We'll store all of the intermediate stages as well, so we can plot the trajectories. See algorithm 18.2"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "A-ZMFOvACIOw"
+ },
+ "outputs": [],
+ "source": [
+ "def sample(model, T, sigma_t, n_samples):\n",
+ " # Create the output array\n",
+ " # Each row represents a time step, first row will be sampled data\n",
+ " # Each column represents a different sample\n",
+ " samples = np.zeros((T+1,n_samples))\n",
+ "\n",
+ " # TODO -- Initialize the samples z_{T} at samples[T,:] from standard normal distribution\n",
+ " # Replace this line\n",
+ " samples[T,:] = np.zeros((1,n_samples))\n",
+ "\n",
+ "\n",
+ "\n",
+ " # For t=100...99..98... ...0\n",
+ " for t in range(T,0,-1):\n",
+ " clear_output(wait=True)\n",
+ " display(\"Predicting z_{%d} from z_{%d}\"%(t-1,t))\n",
+ " # TODO Predict samples[t-1,:] from samples[t,:] using the appropriate model\n",
+ " # Replace this line:\n",
+ " samples[t-1,:] = np.zeros((1,n_samples))\n",
+ "\n",
+ "\n",
+ " # If not the last time step\n",
+ " if t>0:\n",
+ " # TODO Add noise to the samples at z_t-1 we just generated with mean zero, standard deviation sigma_t\n",
+ " # Replace this line\n",
+ " samples[t-1,:] = samples[t-1,:]\n",
+ "\n",
+ " return samples"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ECAUfHNi9NVW"
+ },
+ "source": [
+ "Now let's run the diffusion process for a whole bunch of samples"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "M-TY5w9Q8LYW"
+ },
+ "outputs": [],
+ "source": [
+ "sigma_t=0.12288\n",
+ "n_samples = 100000\n",
+ "samples = sample(all_models, T, sigma_t, n_samples)\n",
+ "\n",
+ "\n",
+ "# Plot the data\n",
+ "sampled_data = samples[0,:]\n",
+ "bins = np.arange(-3,3.05,0.05)\n",
+ "\n",
+ "fig,ax = plt.subplots()\n",
+ "fig.set_size_inches(8,2.5)\n",
+ "ax.set_xlim([-3,3])\n",
+ "plt.hist(sampled_data, bins=bins, density =True)\n",
+ "ax.set_ylim(0, 0.8)\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "jYrAW6tN-gJ4"
+ },
+ "source": [
+ "Let's, plot the evolution of a few of the paths as in figure 18.7 (paths are from bottom to top now)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4XU6CDZC_kFo"
+ },
+ "outputs": [],
+ "source": [
+ "fig, ax = plt.subplots()\n",
+ "t_vals = np.arange(0,101,1)\n",
+ "ax.plot(samples[:,0],t_vals,'r-')\n",
+ "ax.plot(samples[:,1],t_vals,'g-')\n",
+ "ax.plot(samples[:,2],t_vals,'b-')\n",
+ "ax.plot(samples[:,3],t_vals,'c-')\n",
+ "ax.plot(samples[:,4],t_vals,'m-')\n",
+ "ax.set_xlim([-3,3])\n",
+ "ax.set_ylim([101, 0])\n",
+ "ax.set_xlabel('value')\n",
+ "ax.set_ylabel('z_{t}')\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "SGTYGGevAktz"
+ },
+ "source": [
+ "Notice that the samples have a tendency to move from positions that are near the center at time 100 to positions that are high in the true probability distribution at time 0"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "authorship_tag": "ABX9TyNd+D0/IVWXtU2GKsofyk2d",
+ "include_colab_link": true,
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3",
+ "name": "python3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/Notebooks/Chap18/18_4_Families_of_Diffusion_Models.ipynb b/Notebooks/Chap18/18_4_Families_of_Diffusion_Models.ipynb
new file mode 100644
index 0000000..b01d7ff
--- /dev/null
+++ b/Notebooks/Chap18/18_4_Families_of_Diffusion_Models.ipynb
@@ -0,0 +1,494 @@
+{
+ "cells": [
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "colab_type": "text",
+ "id": "view-in-github"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ },
+ "source": [
+ "# **Notebook 18.4: Families of diffusion models**\n",
+ "\n",
+ "This notebook investigates the reparameterized model as described in section 18.5 of the book and computers the results shown in figure 18.10c-f. These models are based on the paper \"Denoising diffusion implicit models\" which can be found [here](https://arxiv.org/pdf/2010.02502.pdf).\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from matplotlib.colors import ListedColormap\n",
+ "from operator import itemgetter\n",
+ "from scipy import stats\n",
+ "from IPython.display import display, clear_output"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4PM8bf6lO0VE"
+ },
+ "outputs": [],
+ "source": [
+ "#Create pretty colormap as in book\n",
+ "my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ "my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ "r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ "g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ "b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ "my_colormap_vals = np.vstack((r,g,b)).transpose()/255.0\n",
+ "my_colormap = ListedColormap(my_colormap_vals)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "ONGRaQscfIOo"
+ },
+ "outputs": [],
+ "source": [
+ "# Probability distribution for normal\n",
+ "def norm_pdf(x, mu, sigma):\n",
+ " return np.exp(-0.5 * (x-mu) * (x-mu) / (sigma * sigma)) / np.sqrt(2*np.pi*sigma*sigma)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "gZvG0MKhfY8Y"
+ },
+ "outputs": [],
+ "source": [
+ "# True distribution is a mixture of four Gaussians\n",
+ "class TrueDataDistribution:\n",
+ " # Constructor initializes parameters\n",
+ " def __init__(self):\n",
+ " self.mu = [1.5, -0.216, 0.45, -1.875]\n",
+ " self.sigma = [0.3, 0.15, 0.525, 0.075]\n",
+ " self.w = [0.2, 0.3, 0.35, 0.15]\n",
+ "\n",
+ " # Return PDF\n",
+ " def pdf(self, x):\n",
+ " return(self.w[0] *norm_pdf(x,self.mu[0],self.sigma[0]) + self.w[1] *norm_pdf(x,self.mu[1],self.sigma[1]) + self.w[2] *norm_pdf(x,self.mu[2],self.sigma[2]) + self.w[3] *norm_pdf(x,self.mu[3],self.sigma[3]))\n",
+ "\n",
+ " # Draw samples\n",
+ " def sample(self, n):\n",
+ " hidden = np.random.choice(4, n, p=self.w)\n",
+ " epsilon = np.random.normal(size=(n))\n",
+ " mu_list = list(itemgetter(*hidden)(self.mu))\n",
+ " sigma_list = list(itemgetter(*hidden)(self.sigma))\n",
+ " return mu_list + sigma_list * epsilon"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "iJu_uBiaeUVv"
+ },
+ "outputs": [],
+ "source": [
+ "# Define ground truth probability distribution that we will model\n",
+ "true_dist = TrueDataDistribution()\n",
+ "# Let's visualize this\n",
+ "x_vals = np.arange(-3,3,0.01)\n",
+ "pr_x_true = true_dist.pdf(x_vals)\n",
+ "fig,ax = plt.subplots()\n",
+ "fig.set_size_inches(8,2.5)\n",
+ "ax.plot(x_vals, pr_x_true, 'r-')\n",
+ "ax.set_xlabel(\"$x$\")\n",
+ "ax.set_ylabel(\"$Pr(x)$\")\n",
+ "ax.set_ylim(0,1.0)\n",
+ "ax.set_xlim(-3,3)\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "DRHUG_41i4t_"
+ },
+ "source": [
+ "To train the model to describe this distribution, we'll need to generate pairs of samples drawn from $Pr(z_t|x)$ (diffusion kernel) and $q(z_{t-1}|z_{t},x)$ (equation 18.15).\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "x6B8t72Ukscd"
+ },
+ "outputs": [],
+ "source": [
+ "# Return z_t (the argument of g_{t}[] in the loss function in algorithm 18.1) and epsilon\n",
+ "def get_data_pairs(x_train,t,beta):\n",
+ "\n",
+ " epsilon = np.random.standard_normal(x_train.shape)\n",
+ " alpha_t = np.power(1-beta,t)\n",
+ " z_t = x_train * np.sqrt(alpha_t) + np.sqrt(1-alpha_t) * epsilon\n",
+ "\n",
+ " return z_t, epsilon"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "aSG_4uA8_zZ-"
+ },
+ "source": [
+ "We also need models $\\text{g}_t[z_{t},\\phi_{t}]$ that map from $z_{t}$ to the noise $\\epsilon$ that was added. We're just going to use a very hacky non-parametric model (basically a lookup table) that tells you the result based on the (quantized) input."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "ZHViC0pL_yy5"
+ },
+ "outputs": [],
+ "source": [
+ "# This code is really ugly! Don't look too closely at it!\n",
+ "# All you need to know is that it is a model that trains from pairs zt, zt_minus1\n",
+ "# And can then predict zt\n",
+ "class NonParametricModel():\n",
+ " # Constructor initializes parameters\n",
+ " def __init__(self):\n",
+ "\n",
+ " self.inc = 0.01\n",
+ " self.max_val = 3.0\n",
+ " self.model = []\n",
+ "\n",
+ " # Learns a model that predicts epsilon given z_t\n",
+ " def train(self, zt, epsilon):\n",
+ " zt = np.clip(zt,-self.max_val,self.max_val)\n",
+ " epsilon = np.clip(epsilon,-self.max_val,self.max_val)\n",
+ " bins = np.arange(-self.max_val,self.max_val+self.inc,self.inc)\n",
+ " numerator, *_ = stats.binned_statistic(zt, epsilon, statistic='sum',bins=bins)\n",
+ " denominator, *_ = stats.binned_statistic(zt, epsilon, statistic='count',bins=bins)\n",
+ " self.model = numerator / (denominator + 1)\n",
+ "\n",
+ " def predict(self, zt):\n",
+ " bin_index = np.floor((zt+self.max_val)/self.inc)\n",
+ " bin_index = np.clip(bin_index,0, len(self.model)-1).astype('uint32')\n",
+ " return self.model[bin_index]"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "CzVFybWoBygu"
+ },
+ "outputs": [],
+ "source": [
+ "# Sample data from distribution (this would usually be our collected training set)\n",
+ "n_sample = 100000\n",
+ "x_train = true_dist.sample(n_sample)\n",
+ "\n",
+ "# Define model parameters\n",
+ "T = 100\n",
+ "beta = 0.01511\n",
+ "\n",
+ "all_models = []\n",
+ "for t in range(0,T):\n",
+ " clear_output(wait=True)\n",
+ " display(\"Training timestep %d\"%(t))\n",
+ " zt,epsilon= get_data_pairs(x_train,t,beta)\n",
+ " all_models.append(NonParametricModel())\n",
+ " # The model at index t maps data from z_{t+1} to epsilon\n",
+ " all_models[t].train(zt,epsilon)"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ZPc9SEvtl14U"
+ },
+ "source": [
+ "Now that we've learned the model, let's draw some samples from it. We start at $z_{100}$ and use the model to predict $z_{99}$, then $z_{98}$ and so on until finally we get to $z_{1}$ and then $x$ (represented as $z_{0}$ here). We'll store all of the intermediate stages as well, so we can plot the trajectories.\n",
+ "\n",
+ "This is the same model we learned last time. The whole point of this is that it is compatible with any forward process with the same diffusion kernel.\n",
+ "\n",
+ "One such model is the denoising diffusion implicit model, which has a sampling step:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "\\mathbf{z}_{t-1} = \\sqrt{\\alpha_{t-1}}\\left(\\frac{\\mathbf{z}_{t}-\\sqrt{1-\\alpha_{t}}\\text{g}_t[\\mathbf{z}_{t},\\boldsymbol\\phi]}{\\sqrt{\\alpha_{t}}}\\right) + \\sqrt{1-\\alpha_{t-1}-\\sigma^2}\\text{g}_t[\\mathbf{z}_{t},\\boldsymbol\\phi]+\\sigma\\epsilon\n",
+ "\\end{equation}\n",
+ "\n",
+ "(see equation 12 of the denoising [diffusion implicit models paper ](https://arxiv.org/pdf/2010.02502.pdf).\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "A-ZMFOvACIOw"
+ },
+ "outputs": [],
+ "source": [
+ "def sample_ddim(model, T, sigma_t, n_samples):\n",
+ " # Create the output array\n",
+ " # Each row represents a time step, first row will be sampled data\n",
+ " # Each column represents a different sample\n",
+ " samples = np.zeros((T+1,n_samples))\n",
+ " samples[T,:] = np.random.standard_normal(n_samples)\n",
+ "\n",
+ " # For t=100...99..98... ...0\n",
+ " for t in range(T,0,-1):\n",
+ " clear_output(wait=True)\n",
+ " display(\"Predicting z_{%d} from z_{%d}\"%(t-1,t))\n",
+ "\n",
+ " alpha_t = np.power(1-beta,t+1)\n",
+ " alpha_t_minus1 = np.power(1-beta,t)\n",
+ "\n",
+ " # TODO -- implement the DDIM sampling step\n",
+ " # Note the final noise term is already added in the \"if\" statement below\n",
+ " # Replace this line:\n",
+ " samples[t-1,:] = samples[t-1,:]\n",
+ "\n",
+ " # If not the last time step\n",
+ " if t>0:\n",
+ " samples[t-1,:] = samples[t-1,:]+ np.random.standard_normal(n_samples) * sigma_t\n",
+ " return samples"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ECAUfHNi9NVW"
+ },
+ "source": [
+ "Now let's run the diffusion process for a whole bunch of samples"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "M-TY5w9Q8LYW"
+ },
+ "outputs": [],
+ "source": [
+ "# Now we'll set the noise to a MUCH smaller level\n",
+ "sigma_t=0.001\n",
+ "n_samples = 100000\n",
+ "samples_low_noise = sample_ddim(all_models, T, sigma_t, n_samples)\n",
+ "\n",
+ "\n",
+ "# Plot the data\n",
+ "sampled_data = samples_low_noise[0,:]\n",
+ "bins = np.arange(-3,3.05,0.05)\n",
+ "\n",
+ "fig,ax = plt.subplots()\n",
+ "fig.set_size_inches(8,2.5)\n",
+ "ax.set_xlim([-3,3])\n",
+ "plt.hist(sampled_data, bins=bins, density =True)\n",
+ "ax.set_ylim(0, 0.8)\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "jYrAW6tN-gJ4"
+ },
+ "source": [
+ "Let's, plot the evolution of a few of the paths as in figure 18.7 (paths are from bottom to top now)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4XU6CDZC_kFo"
+ },
+ "outputs": [],
+ "source": [
+ "fig, ax = plt.subplots()\n",
+ "t_vals = np.arange(0,101,1)\n",
+ "ax.plot(samples_low_noise[:,0],t_vals,'r-')\n",
+ "ax.plot(samples_low_noise[:,1],t_vals,'g-')\n",
+ "ax.plot(samples_low_noise[:,2],t_vals,'b-')\n",
+ "ax.plot(samples_low_noise[:,3],t_vals,'c-')\n",
+ "ax.plot(samples_low_noise[:,4],t_vals,'m-')\n",
+ "ax.set_xlim([-3,3])\n",
+ "ax.set_ylim([101, 0])\n",
+ "ax.set_xlabel('value')\n",
+ "ax.set_ylabel('z_{t}')\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "SGTYGGevAktz"
+ },
+ "source": [
+ "The samples have a tendency to move from positions that are near the center at time 100 to positions that are high in the true probability distribution at time 0"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Z-LZp_fMXxRt"
+ },
+ "source": [
+ "Let's now sample from the accelerated model, that requires fewer models. Again, we don't need to learn anything new -- this is just the reverse process that corresponds to a different forward process that is compatible with the same diffusion kernel.\n",
+ "\n",
+ "There's nothing to do here except read the code. It uses the same DDIM model as you just implemented in the previous step, but it jumps timesteps five at a time."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "3Z0erjGbYj1u"
+ },
+ "outputs": [],
+ "source": [
+ "def sample_accelerated(model, T, sigma_t, n_steps, n_samples):\n",
+ " # Create the output array\n",
+ " # Each row represents a sample (i.e. fewer than the time steps), first row will be sampled data\n",
+ " # Each column represents a different sample\n",
+ " samples = np.zeros((n_steps+1,n_samples))\n",
+ " samples[n_steps,:] = np.random.standard_normal(n_samples)\n",
+ "\n",
+ " # For each sampling step\n",
+ " for c_step in range(n_steps,0,-1):\n",
+ " # Find the corresponding time step and previous time step\n",
+ " t= int(T * c_step/n_steps)\n",
+ " tminus1 = int(T * (c_step-1)/n_steps)\n",
+ " display(\"Predicting z_{%d} from z_{%d}\"%(tminus1,t))\n",
+ "\n",
+ " alpha_t = np.power(1-beta,t+1)\n",
+ " alpha_t_minus1 = np.power(1-beta,tminus1+1)\n",
+ " epsilon_est = all_models[t-1].predict(samples[c_step,:])\n",
+ "\n",
+ " samples[c_step-1,:]=np.sqrt(alpha_t_minus1)*(samples[c_step,:]-np.sqrt(1-alpha_t) * epsilon_est)/np.sqrt(alpha_t) \\\n",
+ " + np.sqrt(1-alpha_t_minus1 - sigma_t*sigma_t) * epsilon_est\n",
+ " # If not the last time step\n",
+ " if t>0:\n",
+ " samples[c_step-1,:] = samples[c_step-1,:]+ np.random.standard_normal(n_samples) * sigma_t\n",
+ " return samples"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "D3Sm_WYrcuED"
+ },
+ "source": [
+ "Now let's draw a bunch of samples from the model"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "UB45c7VMcGy-"
+ },
+ "outputs": [],
+ "source": [
+ "sigma_t=0.11\n",
+ "n_samples = 100000\n",
+ "n_steps = 20 # i.e. sample 5 times as fast as before -- should be a divisor of 100\n",
+ "samples_accelerated = sample_accelerated(all_models, T, sigma_t, n_steps, n_samples)\n",
+ "\n",
+ "\n",
+ "# Plot the data\n",
+ "sampled_data = samples_accelerated[0,:]\n",
+ "bins = np.arange(-3,3.05,0.05)\n",
+ "\n",
+ "fig,ax = plt.subplots()\n",
+ "fig.set_size_inches(8,2.5)\n",
+ "ax.set_xlim([-3,3])\n",
+ "plt.hist(sampled_data, bins=bins, density =True)\n",
+ "ax.set_ylim(0, 0.9)\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Luv-6w84c_qO"
+ },
+ "outputs": [],
+ "source": [
+ "fig, ax = plt.subplots()\n",
+ "step_increment = 100/ n_steps\n",
+ "t_vals = np.arange(0,101,5)\n",
+ "\n",
+ "for i in range(len(t_vals)-1):\n",
+ " ax.plot( (samples_accelerated[i,0],samples_accelerated[i+1,0]), (t_vals[i], t_vals[i+1]),'r.-')\n",
+ " ax.plot( (samples_accelerated[i,1],samples_accelerated[i+1,1]), (t_vals[i], t_vals[i+1]),'g.-')\n",
+ " ax.plot( (samples_accelerated[i,2],samples_accelerated[i+1,2]), (t_vals[i], t_vals[i+1]),'b.-')\n",
+ " ax.plot( (samples_accelerated[i,3],samples_accelerated[i+1,3]), (t_vals[i], t_vals[i+1]),'c.-')\n",
+ " ax.plot( (samples_accelerated[i,4],samples_accelerated[i+1,4]), (t_vals[i], t_vals[i+1]),'m.-')\n",
+ "\n",
+ "ax.set_xlim([-3,3])\n",
+ "ax.set_ylim([101, 0])\n",
+ "ax.set_xlabel('value')\n",
+ "ax.set_ylabel('z_{t}')\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "LSJi72f0kw_e"
+ },
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "authorship_tag": "ABX9TyNFSvISBXo/Z1l+onknF2Gw",
+ "include_colab_link": true,
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3",
+ "name": "python3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/Notebooks/Chap19/19_1_Markov_Decision_Processes.ipynb b/Notebooks/Chap19/19_1_Markov_Decision_Processes.ipynb
new file mode 100644
index 0000000..ddac7cc
--- /dev/null
+++ b/Notebooks/Chap19/19_1_Markov_Decision_Processes.ipynb
@@ -0,0 +1,736 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyPg3umHnqmIXX6jGe809Nxf",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 19.1: Markov Decision Processes**\n",
+ "\n",
+ "This notebook investigates Markov decision processes as described in section 19.1 of the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from PIL import Image"
+ ],
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Get local copies of components of images\n",
+ "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Empty.png\n",
+ "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Hole.png\n",
+ "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Fish.png\n",
+ "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Penguin.png"
+ ],
+ "metadata": {
+ "id": "ZsvrUszPLyEG"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Ugly class that takes care of drawing pictures like in the book.\n",
+ "# You can totally ignore this code!\n",
+ "class DrawMDP:\n",
+ " # Constructor initializes parameters\n",
+ " def __init__(self, n_row, n_col):\n",
+ " self.empty_image = np.asarray(Image.open('Empty.png'))\n",
+ " self.hole_image = np.asarray(Image.open('Hole.png'))\n",
+ " self.fish_image = np.asarray(Image.open('Fish.png'))\n",
+ " self.penguin_image = np.asarray(Image.open('Penguin.png'))\n",
+ " self.fig,self.ax = plt.subplots()\n",
+ " self.n_row = n_row\n",
+ " self.n_col = n_col\n",
+ "\n",
+ " my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ " my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ " r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ " g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ " b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ " self.colormap = np.vstack((r,g,b)).transpose()/255.0\n",
+ "\n",
+ "\n",
+ " def draw_text(self, text, row, col, position, color):\n",
+ " if position == 'bc':\n",
+ " self.ax.text( 83*col+41,83 * (row+1) -10, text, horizontalalignment=\"center\", color=color, fontweight='bold')\n",
+ " if position == 'tl':\n",
+ " self.ax.text( 83*col+5,83 * row +5, text, verticalalignment = 'top', horizontalalignment=\"left\", color=color, fontweight='bold')\n",
+ "\n",
+ " # Draws a set of states\n",
+ " def draw_path(self, path, color1, color2):\n",
+ " for i in range(len(path)-1):\n",
+ " row_start = np.floor(path[i]/self.n_col)\n",
+ " row_end = np.floor(path[i+1]/self.n_col)\n",
+ " col_start = path[i] - row_start * self.n_col\n",
+ " col_end = path[i+1] - row_end * self.n_col\n",
+ "\n",
+ " color_index = int(np.floor(255 * i/(len(path)-1.)))\n",
+ " self.ax.plot([col_start * 83+41 + i, col_end * 83+41 + i ],[row_start * 83+41 + i, row_end * 83+41 + i ], color=(self.colormap[color_index,0],self.colormap[color_index,1],self.colormap[color_index,2]))\n",
+ "\n",
+ "\n",
+ " # Draw deterministic policy\n",
+ " def draw_deterministic_policy(self,i, action):\n",
+ " row = np.floor(i/self.n_col)\n",
+ " col = i - row * self.n_col\n",
+ " center_x = 83 * col + 41\n",
+ " center_y = 83 * row + 41\n",
+ " arrow_base_width = 10\n",
+ " arrow_height = 15\n",
+ " # Draw arrow pointing upward\n",
+ " if action ==0:\n",
+ " triangle_indices = np.array([[center_x, center_y-arrow_height/2],\n",
+ " [center_x - arrow_base_width/2, center_y+arrow_height/2],\n",
+ " [center_x + arrow_base_width/2, center_y+arrow_height/2]])\n",
+ " # Draw arrow pointing right\n",
+ " if action ==1:\n",
+ " triangle_indices = np.array([[center_x + arrow_height/2, center_y],\n",
+ " [center_x - arrow_height/2, center_y-arrow_base_width/2],\n",
+ " [center_x - arrow_height/2, center_y+arrow_base_width/2]])\n",
+ " # Draw arrow pointing downward\n",
+ " if action ==2:\n",
+ " triangle_indices = np.array([[center_x, center_y+arrow_height/2],\n",
+ " [center_x - arrow_base_width/2, center_y-arrow_height/2],\n",
+ " [center_x + arrow_base_width/2, center_y-arrow_height/2]])\n",
+ " # Draw arrow pointing left\n",
+ " if action ==3:\n",
+ " triangle_indices = np.array([[center_x - arrow_height/2, center_y],\n",
+ " [center_x + arrow_height/2, center_y-arrow_base_width/2],\n",
+ " [center_x + arrow_height/2, center_y+arrow_base_width/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ " # Draw stochastic policy\n",
+ " def draw_stochastic_policy(self,i, action_probs):\n",
+ " row = np.floor(i/self.n_col)\n",
+ " col = i - row * self.n_col\n",
+ " offset = 20\n",
+ " # Draw arrow pointing upward\n",
+ " center_x = 83 * col + 41\n",
+ " center_y = 83 * row + 41 - offset\n",
+ " arrow_base_width = 15 * action_probs[0]\n",
+ " arrow_height = 20 * action_probs[0]\n",
+ " triangle_indices = np.array([[center_x, center_y-arrow_height/2],\n",
+ " [center_x - arrow_base_width/2, center_y+arrow_height/2],\n",
+ " [center_x + arrow_base_width/2, center_y+arrow_height/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ " # Draw arrow pointing right\n",
+ " center_x = 83 * col + 41 + offset\n",
+ " center_y = 83 * row + 41\n",
+ " arrow_base_width = 15 * action_probs[1]\n",
+ " arrow_height = 20 * action_probs[1]\n",
+ " triangle_indices = np.array([[center_x + arrow_height/2, center_y],\n",
+ " [center_x - arrow_height/2, center_y-arrow_base_width/2],\n",
+ " [center_x - arrow_height/2, center_y+arrow_base_width/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ " # Draw arrow pointing downward\n",
+ " center_x = 83 * col + 41\n",
+ " center_y = 83 * row + 41 +offset\n",
+ " arrow_base_width = 15 * action_probs[2]\n",
+ " arrow_height = 20 * action_probs[2]\n",
+ " triangle_indices = np.array([[center_x, center_y+arrow_height/2],\n",
+ " [center_x - arrow_base_width/2, center_y-arrow_height/2],\n",
+ " [center_x + arrow_base_width/2, center_y-arrow_height/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ " # Draw arrow pointing left\n",
+ " center_x = 83 * col + 41 -offset\n",
+ " center_y = 83 * row + 41\n",
+ " arrow_base_width = 15 * action_probs[3]\n",
+ " arrow_height = 20 * action_probs[3]\n",
+ " triangle_indices = np.array([[center_x - arrow_height/2, center_y],\n",
+ " [center_x + arrow_height/2, center_y-arrow_base_width/2],\n",
+ " [center_x + arrow_height/2, center_y+arrow_base_width/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ " def draw(self, layout, state, draw_state_index= False, rewards=None, policy=None, state_values=None, action_values=None,path1=None, path2 = None):\n",
+ " # Construct the image\n",
+ " image_out = np.zeros((self.n_row * 83, self.n_col * 83, 4),dtype='uint8')\n",
+ " for c_row in range (self.n_row):\n",
+ " for c_col in range(self.n_col):\n",
+ " if layout[c_row * self.n_col + c_col]==0:\n",
+ " image_out[c_row*83:c_row*83+83, c_col*83:c_col*83+83,:] = self.empty_image\n",
+ " elif layout[c_row * self.n_col + c_col]==1:\n",
+ " image_out[c_row*83:c_row*83+83, c_col*83:c_col*83+83,:] = self.hole_image\n",
+ " else:\n",
+ " image_out[c_row*83:c_row*83+83, c_col*83:c_col*83+83,:] = self.fish_image\n",
+ " if state == c_row * self.n_col + c_col:\n",
+ " image_out[c_row*83:c_row*83+83, c_col*83:c_col*83+83,:] = self.penguin_image\n",
+ "\n",
+ " # Draw the image\n",
+ " plt.imshow(image_out)\n",
+ " self.ax.get_xaxis().set_visible(False)\n",
+ " self.ax.get_yaxis().set_visible(False)\n",
+ " self.ax.spines['top'].set_visible(False)\n",
+ " self.ax.spines['right'].set_visible(False)\n",
+ " self.ax.spines['bottom'].set_visible(False)\n",
+ " self.ax.spines['left'].set_visible(False)\n",
+ "\n",
+ " if draw_state_index:\n",
+ " for c_cell in range(layout.size):\n",
+ " self.draw_text(\"%d\"%(c_cell), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'tl','k')\n",
+ "\n",
+ " # Draw the policy as triangles\n",
+ " if policy is not None:\n",
+ " # If the policy is deterministic\n",
+ " if len(policy) == len(layout):\n",
+ " for i in range(len(layout)):\n",
+ " self.draw_deterministic_policy(i, policy[i])\n",
+ " # Else it is stochastic\n",
+ " else:\n",
+ " for i in range(len(layout)):\n",
+ " self.draw_stochastic_policy(i,policy[:,i])\n",
+ "\n",
+ "\n",
+ " if path1 is not None:\n",
+ " # self.draw_path(path1, np.array([0.81, 0.51, 0.38]), np.array([1.0, 0.2, 0.5]))\n",
+ " self.draw_path(path1, np.array([1.0, 0.0, 0.0]), np.array([0.0, 1.0, 1.0]))\n",
+ "\n",
+ "\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "Gq1HfJsHN3SB"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's draw the initial situation with the penguin in top right\n",
+ "n_rows = 4; n_cols = 4\n",
+ "layout = np.zeros(n_rows * n_cols)\n",
+ "initial_state = 0\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, state = initial_state, draw_state_index = True)"
+ ],
+ "metadata": {
+ "id": "eBQ7lTpJQBSe"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Note that the states are indexed from 0 rather than 1 as in the book to make\n",
+ "the code neater."
+ ],
+ "metadata": {
+ "id": "P7P40UyMunKb"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Define the state probabilities\n",
+ "transition_probabilities = np.array( \\\n",
+ "[[0.00 , 0.33, 0.00, 0.00, 0.33, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.50 , 0.00, 0.33, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.33, 0.00, 0.50, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.33, 0.00, 0.00, 0.00, 0.00, 0.33, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.50 , 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.33, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.34, 0.00, 0.00, 0.33, 0.00, 0.25, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.34, 0.00, 0.00, 0.25, 0.00, 0.33, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.50, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.33, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.34, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.33, 0.00, 0.25, 0.00, 0.00, 0.33, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.25, 0.00, 0.33, 0.00, 0.00, 0.33, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.34, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.50 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.34, 0.00, 0.00, 0.00, 0.00, 0.33, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.50, 0.00, 0.33, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.34, 0.00, 0.50 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.34, 0.00, 0.00, 0.34, 0.00 ],\n",
+ "])\n",
+ "initial_state = 0"
+ ],
+ "metadata": {
+ "id": "wgFcIi4YQJWI"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define a step from the Markov process"
+ ],
+ "metadata": {
+ "id": "axllRDDuDDLS"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def markov_process_step(state, transition_probabilities):\n",
+ " # TODO -- update the state according to the appropriate transition probabilities\n",
+ " # One way to do this is to use np.random.choice\n",
+ " # Replace this line:\n",
+ " new_state = 0\n",
+ "\n",
+ "\n",
+ " return new_state"
+ ],
+ "metadata": {
+ "id": "FrSZrS67sdbN"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Run the Markov process for 10 steps and visualise the results"
+ ],
+ "metadata": {
+ "id": "uTj7rN6LDFXd"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "np.random.seed(0)\n",
+ "T = 10\n",
+ "states = np.zeros(T, dtype='uint8')\n",
+ "states[0] = 0\n",
+ "for t in range(T-1):\n",
+ " states[t+1] = markov_process_step(states[t], transition_probabilities)\n",
+ "\n",
+ "\n",
+ "\n",
+ "print(\"Your States:\", states)\n",
+ "print(\"True States: [ 0 4 8 9 10 9 10 9 13 14]\")\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, state = states[0], path1=states, draw_state_index = True)"
+ ],
+ "metadata": {
+ "id": "lRIdjagCwP62"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define a Markov one step of a reward process."
+ ],
+ "metadata": {
+ "id": "QLyjyBjjDMin"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def markov_reward_process_step(state, transition_probabilities, reward_structure):\n",
+ "\n",
+ " # TODO -- write this function\n",
+ " # Update the state. Return a reward of +1 if the Penguin lands on the fish\n",
+ " # or zero otherwise.\n",
+ " # Replace this line\n",
+ " new_state = 0; reward = 0\n",
+ "\n",
+ "\n",
+ " return new_state, reward"
+ ],
+ "metadata": {
+ "id": "YPHSJRKx-pgO"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Run the Markov reward process for 10 steps and visualise the results"
+ ],
+ "metadata": {
+ "id": "AIz8QEiRFoCm"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set up the reward structure so it matches figure 19.2\n",
+ "reward_structure = np.zeros((16,1))\n",
+ "reward_structure[3] = 1; reward_structure[8] = 1; reward_structure[10] = 1\n",
+ "\n",
+ "# Initialize random numbers\n",
+ "np.random.seed(0)\n",
+ "T = 10\n",
+ "# Set up the states, so the fish are in the same positions as figure 19.2\n",
+ "states = np.zeros(T, dtype='uint8')\n",
+ "rewards = np.zeros(T, dtype='uint8')\n",
+ "\n",
+ "states[0] = 0\n",
+ "for t in range(T-1):\n",
+ " states[t+1],rewards[t+1] = markov_reward_process_step(states[t], transition_probabilities, reward_structure)\n",
+ "\n",
+ "print(\"Your States:\", states)\n",
+ "print(\"Your Rewards:\", rewards)\n",
+ "print(\"True Rewards: [0 0 1 0 1 0 1 0 0 0]\")\n",
+ "\n",
+ "\n",
+ "# Draw the figure\n",
+ "layout = np.zeros(n_rows * n_cols)\n",
+ "layout[3] = 2; layout[8] = 2 ; layout[10] = 2\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, state = states[0], path1=states, draw_state_index = True)"
+ ],
+ "metadata": {
+ "id": "0p1gCpGoFn4M"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's calculate the return -- the sum of discounted future rewards"
+ ],
+ "metadata": {
+ "id": "lyz47NWrITfj"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def calculate_return(rewards, gamma):\n",
+ " # TODO -- you write this function\n",
+ " # It should compute one return for the start of the sequence (i.e. G_1)\n",
+ " # Replace this line\n",
+ " return_val = 0.0\n",
+ "\n",
+ "\n",
+ " return return_val"
+ ],
+ "metadata": {
+ "id": "4fEuBRPnFm_N"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "gamma = 0.9\n",
+ "for t in range(len(states)):\n",
+ " print(\"Return at time %d = %3.3f\"%(t, calculate_return(rewards[t:],gamma)))\n",
+ "\n",
+ "# Reality check!\n",
+ "print(\"True return at time 0: 1.998\")"
+ ],
+ "metadata": {
+ "id": "o19lQgM3JrOz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's define the state transition function $Pr(s_{t+1}|s_{t},a)$ in full where $a$ is the actions. Here $a=0$ means try to go upward, $a=1$, right, $a=2$ down and $a=3$ right. However, the ice is slippery, so we don't always go the direction we want to.\n",
+ "\n",
+ "Note that as for the states, we've indexed the actions from zero (unlike in the book, so they map to the indices of arrays better)"
+ ],
+ "metadata": {
+ "id": "Fhc6DzZNOjiC"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "transition_probabilities_given_action1 = np.array(\\\n",
+ "[[0.00 , 0.33, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.50 , 0.00, 0.33, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.33, 0.00, 0.50, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.33, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.50 , 0.00, 0.00, 0.00, 0.00, 0.17, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.34, 0.00, 0.00, 0.25, 0.00, 0.17, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.34, 0.00, 0.00, 0.17, 0.00, 0.25, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.50, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.17, 0.00, 0.00, 0.75, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.25, 0.00, 0.17, 0.00, 0.00, 0.50, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.17, 0.00, 0.25, 0.00, 0.00, 0.50, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.75 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.25, 0.00, 0.25, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.25, 0.00, 0.25 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.25, 0.00 ],\n",
+ "])\n",
+ "\n",
+ "transition_probabilities_given_action2 = np.array(\\\n",
+ "[[0.00 , 0.25, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.75 , 0.00, 0.25, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.50, 0.00, 0.50, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.33, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.25 , 0.00, 0.00, 0.00, 0.00, 0.17, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.25, 0.00, 0.00, 0.50, 0.00, 0.17, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.25, 0.00, 0.00, 0.50, 0.00, 0.33, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.50, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.33, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.17, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.50, 0.00, 0.17, 0.00, 0.00, 0.25, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.50, 0.00, 0.33, 0.00, 0.00, 0.25, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.34, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.50 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.75, 0.00, 0.25, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.50, 0.00, 0.50 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.34, 0.00, 0.00, 0.50, 0.00 ],\n",
+ "])\n",
+ "\n",
+ "transition_probabilities_given_action3 = np.array(\\\n",
+ "[[0.00 , 0.25, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.25 , 0.00, 0.25, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.25, 0.00, 0.25, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.75 , 0.00, 0.00, 0.00, 0.00, 0.17, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.50, 0.00, 0.00, 0.25, 0.00, 0.17, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.50, 0.00, 0.00, 0.16, 0.00, 0.25, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.75, 0.00, 0.00, 0.16, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.17, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.25, 0.00, 0.17, 0.00, 0.00, 0.33, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.16, 0.00, 0.25, 0.00, 0.00, 0.33, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.16, 0.00, 0.00, 0.00, 0.00, 0.50 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.33, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.50, 0.00, 0.33, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.34, 0.00, 0.50 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.34, 0.00 ],\n",
+ "])\n",
+ "\n",
+ "transition_probabilities_given_action4 = np.array(\\\n",
+ "[[0.00 , 0.25, 0.00, 0.00, 0.33, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.50 , 0.00, 0.25, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.50, 0.00, 0.75, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.50 , 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.33, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.25, 0.00, 0.00, 0.33, 0.00, 0.50, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.25, 0.00, 0.00, 0.17, 0.00, 0.50, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.25, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.34, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.33, 0.00, 0.50, 0.00, 0.00, 0.25, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.17, 0.00, 0.50, 0.00, 0.00, 0.25, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.25 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.34, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.50, 0.00, 0.50, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.25, 0.00, 0.75 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.25, 0.00 ],\n",
+ "])\n",
+ "\n",
+ "# Store all of these in a three dimension array\n",
+ "# Pr(s_{t+1}=2|s_{t}=1, a_{t}=3] is stored at position [2,1,3]\n",
+ "transition_probabilities_given_action = np.concatenate((np.expand_dims(transition_probabilities_given_action1,2),\n",
+ " np.expand_dims(transition_probabilities_given_action2,2),\n",
+ " np.expand_dims(transition_probabilities_given_action3,2),\n",
+ " np.expand_dims(transition_probabilities_given_action4,2)),axis=2)"
+ ],
+ "metadata": {
+ "id": "l7rT78BbOgTi"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Now we need a policy. Let's start with the deterministic policy in figure 19.5a:\n",
+ "policy = [2,2,1,1, 2,1,1,1, 1,1,0,2, 1,0,1,1]\n",
+ "\n",
+ "# Let's draw the policy first\n",
+ "layout = np.zeros(n_rows * n_cols)\n",
+ "layout[15] = 2\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, state = states[0], policy = policy, draw_state_index = True)"
+ ],
+ "metadata": {
+ "id": "8jWhDlkaKj7Q"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def markov_decision_process_step_deterministic(state, transition_probabilities_given_action, reward_structure, policy):\n",
+ " # TODO -- complete this function.\n",
+ " # For each state, there's is a corresponding action.\n",
+ " # Draw the next state based on the current state and that action\n",
+ " # and calculate the reward\n",
+ " # Replace this line:\n",
+ " new_state = 0; reward = 0;\n",
+ "\n",
+ " return new_state, reward\n"
+ ],
+ "metadata": {
+ "id": "dueNbS2SUVUK"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set up the reward structure so it matches figure 19.2\n",
+ "reward_structure = np.zeros((16,1))\n",
+ "reward_structure[15] = 1\n",
+ "\n",
+ "# Initialize random number seed\n",
+ "np.random.seed(3)\n",
+ "T = 10\n",
+ "# Set up the states, so the fish are in the same positions as figure 19.5\n",
+ "states = np.zeros(T, dtype='uint8')\n",
+ "rewards = np.zeros(T, dtype='uint8')\n",
+ "\n",
+ "states[0] = 0\n",
+ "for t in range(T-1):\n",
+ " states[t+1],rewards[t+1] = markov_decision_process_step_deterministic(states[t], transition_probabilities_given_action, reward_structure, policy)\n",
+ "\n",
+ "print(\"Your States:\", states)\n",
+ "print(\"True States: [ 0 4 8 9 13 14 15 11 7 3]\")\n",
+ "print(\"Your Rewards:\", rewards)\n",
+ "print(\"True Rewards: [0 0 0 0 0 0 1 0 0 0]\")\n",
+ "\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, state = states[0], path1=states, policy = policy, draw_state_index = True)"
+ ],
+ "metadata": {
+ "id": "4Du5aUfd2Lci"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "You can see that the Penguin usually follows the policy, (heads in the direction of the cyan arrows (when it can). But sometimes, the penguin \"slips\" to a different neighboring state\n",
+ "\n",
+ "Now let's investigate a stochastic policy"
+ ],
+ "metadata": {
+ "id": "bLEd8xug33b-"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "np.random.seed(0)\n",
+ "# Let's now choose a random policy. We'll generate a set of random numbers and pass\n",
+ "# them through a softmax function\n",
+ "stochastic_policy = np.random.normal(size=(4,n_rows*n_cols))\n",
+ "stochastic_policy = np.exp(stochastic_policy) / (np.ones((4,1))@ np.expand_dims(np.sum(np.exp(stochastic_policy), axis=0),0))\n",
+ "np.set_printoptions(precision=2)\n",
+ "print(stochastic_policy)\n",
+ "\n",
+ "# Let's draw the policy first\n",
+ "layout = np.zeros(n_rows * n_cols)\n",
+ "layout[15] = 2\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, state = states[0], path1=states, policy = stochastic_policy, draw_state_index = True)"
+ ],
+ "metadata": {
+ "id": "o7T0b3tyilDc"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def markov_decision_process_step_stochastic(state, transition_probabilities_given_action, reward_structure, stochastic_policy):\n",
+ " # TODO -- complete this function.\n",
+ " # For each state, there's is a corresponding distribution over actions\n",
+ " # Draw a sample from that distribution to get the action\n",
+ " # Draw the next state based on the current state and that action\n",
+ " # and calculate the reward\n",
+ " # Replace this line:\n",
+ " new_state = 0; reward = 0;action = 0\n",
+ "\n",
+ "\n",
+ "\n",
+ " return new_state, reward, action"
+ ],
+ "metadata": {
+ "id": "T68mTZSe6A3w"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set up the reward structure so it matches figure 19.2\n",
+ "reward_structure = np.zeros((16,1))\n",
+ "reward_structure[15] = 1\n",
+ "\n",
+ "# Initialize random number seed\n",
+ "np.random.seed(0)\n",
+ "T = 10\n",
+ "# Set up the states, so the fish are in the same positions as figure 19.5\n",
+ "states = np.zeros(T, dtype='uint8')\n",
+ "rewards = np.zeros(T, dtype='uint8')\n",
+ "actions = np.zeros(T-1, dtype='uint8')\n",
+ "\n",
+ "states[0] = 0\n",
+ "for t in range(T-1):\n",
+ " states[t+1],rewards[t+1],actions[t] = markov_decision_process_step_stochastic(states[t], transition_probabilities_given_action, reward_structure, stochastic_policy)\n",
+ "\n",
+ "print(\"Actions\", actions)\n",
+ "print(\"Your States:\", states)\n",
+ "print(\"Your Rewards:\", rewards)\n",
+ "\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, state = states[0], path1=states, policy = stochastic_policy, draw_state_index = True)"
+ ],
+ "metadata": {
+ "id": "hMRVYX2HtqMg"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
diff --git a/Notebooks/Chap19/19_2_Dynamic_Programming.ipynb b/Notebooks/Chap19/19_2_Dynamic_Programming.ipynb
new file mode 100644
index 0000000..a1a2859
--- /dev/null
+++ b/Notebooks/Chap19/19_2_Dynamic_Programming.ipynb
@@ -0,0 +1,530 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyOlD6kmCxX3SKKuh3oJikKA",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 19.2: Dynamic programming**\n",
+ "\n",
+ "This notebook investigates the dynamic programming approach to tabular reinforcement learning as described in figure 19.10 of the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from PIL import Image"
+ ],
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Get local copies of components of images\n",
+ "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Empty.png\n",
+ "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Hole.png\n",
+ "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Fish.png\n",
+ "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Penguin.png"
+ ],
+ "metadata": {
+ "id": "ZsvrUszPLyEG"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Ugly class that takes care of drawing pictures like in the book.\n",
+ "# You can totally ignore this code!\n",
+ "class DrawMDP:\n",
+ " # Constructor initializes parameters\n",
+ " def __init__(self, n_row, n_col):\n",
+ " self.empty_image = np.asarray(Image.open('Empty.png'))\n",
+ " self.hole_image = np.asarray(Image.open('Hole.png'))\n",
+ " self.fish_image = np.asarray(Image.open('Fish.png'))\n",
+ " self.penguin_image = np.asarray(Image.open('Penguin.png'))\n",
+ " self.fig,self.ax = plt.subplots()\n",
+ " self.n_row = n_row\n",
+ " self.n_col = n_col\n",
+ "\n",
+ " my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ " my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ " r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ " g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ " b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ " self.colormap = np.vstack((r,g,b)).transpose()/255.0\n",
+ "\n",
+ "\n",
+ " def draw_text(self, text, row, col, position, color):\n",
+ " if position == 'bc':\n",
+ " self.ax.text( 83*col+41,83 * (row+1) -10, text, horizontalalignment=\"center\", color=color, fontweight='bold')\n",
+ " if position == 'tl':\n",
+ " self.ax.text( 83*col+5,83 * row +5, text, verticalalignment = 'top', horizontalalignment=\"left\", color=color, fontweight='bold')\n",
+ " if position == 'tr':\n",
+ " self.ax.text( 83*(col+1)-5, 83 * row +5, text, verticalalignment = 'top', horizontalalignment=\"right\", color=color, fontweight='bold')\n",
+ "\n",
+ " # Draws a set of states\n",
+ " def draw_path(self, path, color1, color2):\n",
+ " for i in range(len(path)-1):\n",
+ " row_start = np.floor(path[i]/self.n_col)\n",
+ " row_end = np.floor(path[i+1]/self.n_col)\n",
+ " col_start = path[i] - row_start * self.n_col\n",
+ " col_end = path[i+1] - row_end * self.n_col\n",
+ "\n",
+ " color_index = int(np.floor(255 * i/(len(path)-1.)))\n",
+ " self.ax.plot([col_start * 83+41 + i, col_end * 83+41 + i ],[row_start * 83+41 + i, row_end * 83+41 + i ], color=(self.colormap[color_index,0],self.colormap[color_index,1],self.colormap[color_index,2]))\n",
+ "\n",
+ "\n",
+ " # Draw deterministic policy\n",
+ " def draw_deterministic_policy(self,i, action):\n",
+ " row = np.floor(i/self.n_col)\n",
+ " col = i - row * self.n_col\n",
+ " center_x = 83 * col + 41\n",
+ " center_y = 83 * row + 41\n",
+ " arrow_base_width = 10\n",
+ " arrow_height = 15\n",
+ " # Draw arrow pointing upward\n",
+ " if action ==0:\n",
+ " triangle_indices = np.array([[center_x, center_y-arrow_height/2],\n",
+ " [center_x - arrow_base_width/2, center_y+arrow_height/2],\n",
+ " [center_x + arrow_base_width/2, center_y+arrow_height/2]])\n",
+ " # Draw arrow pointing right\n",
+ " if action ==1:\n",
+ " triangle_indices = np.array([[center_x + arrow_height/2, center_y],\n",
+ " [center_x - arrow_height/2, center_y-arrow_base_width/2],\n",
+ " [center_x - arrow_height/2, center_y+arrow_base_width/2]])\n",
+ " # Draw arrow pointing downward\n",
+ " if action ==2:\n",
+ " triangle_indices = np.array([[center_x, center_y+arrow_height/2],\n",
+ " [center_x - arrow_base_width/2, center_y-arrow_height/2],\n",
+ " [center_x + arrow_base_width/2, center_y-arrow_height/2]])\n",
+ " # Draw arrow pointing left\n",
+ " if action ==3:\n",
+ " triangle_indices = np.array([[center_x - arrow_height/2, center_y],\n",
+ " [center_x + arrow_height/2, center_y-arrow_base_width/2],\n",
+ " [center_x + arrow_height/2, center_y+arrow_base_width/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ " # Draw stochastic policy\n",
+ " def draw_stochastic_policy(self,i, action_probs):\n",
+ " row = np.floor(i/self.n_col)\n",
+ " col = i - row * self.n_col\n",
+ " offset = 20\n",
+ " # Draw arrow pointing upward\n",
+ " center_x = 83 * col + 41\n",
+ " center_y = 83 * row + 41 - offset\n",
+ " arrow_base_width = 15 * action_probs[0]\n",
+ " arrow_height = 20 * action_probs[0]\n",
+ " triangle_indices = np.array([[center_x, center_y-arrow_height/2],\n",
+ " [center_x - arrow_base_width/2, center_y+arrow_height/2],\n",
+ " [center_x + arrow_base_width/2, center_y+arrow_height/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ " # Draw arrow pointing right\n",
+ " center_x = 83 * col + 41 + offset\n",
+ " center_y = 83 * row + 41\n",
+ " arrow_base_width = 15 * action_probs[1]\n",
+ " arrow_height = 20 * action_probs[1]\n",
+ " triangle_indices = np.array([[center_x + arrow_height/2, center_y],\n",
+ " [center_x - arrow_height/2, center_y-arrow_base_width/2],\n",
+ " [center_x - arrow_height/2, center_y+arrow_base_width/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ " # Draw arrow pointing downward\n",
+ " center_x = 83 * col + 41\n",
+ " center_y = 83 * row + 41 +offset\n",
+ " arrow_base_width = 15 * action_probs[2]\n",
+ " arrow_height = 20 * action_probs[2]\n",
+ " triangle_indices = np.array([[center_x, center_y+arrow_height/2],\n",
+ " [center_x - arrow_base_width/2, center_y-arrow_height/2],\n",
+ " [center_x + arrow_base_width/2, center_y-arrow_height/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ " # Draw arrow pointing left\n",
+ " center_x = 83 * col + 41 -offset\n",
+ " center_y = 83 * row + 41\n",
+ " arrow_base_width = 15 * action_probs[3]\n",
+ " arrow_height = 20 * action_probs[3]\n",
+ " triangle_indices = np.array([[center_x - arrow_height/2, center_y],\n",
+ " [center_x + arrow_height/2, center_y-arrow_base_width/2],\n",
+ " [center_x + arrow_height/2, center_y+arrow_base_width/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ "\n",
+ " def draw(self, layout, state=None, draw_state_index= False, rewards=None, policy=None, state_values=None, action_values=None,path1=None, path2 = None):\n",
+ " # Construct the image\n",
+ " image_out = np.zeros((self.n_row * 83, self.n_col * 83, 4),dtype='uint8')\n",
+ " for c_row in range (self.n_row):\n",
+ " for c_col in range(self.n_col):\n",
+ " if layout[c_row * self.n_col + c_col]==0:\n",
+ " image_out[c_row*83:c_row*83+83, c_col*83:c_col*83+83,:] = self.empty_image\n",
+ " elif layout[c_row * self.n_col + c_col]==1:\n",
+ " image_out[c_row*83:c_row*83+83, c_col*83:c_col*83+83,:] = self.hole_image\n",
+ " else:\n",
+ " image_out[c_row*83:c_row*83+83, c_col*83:c_col*83+83,:] = self.fish_image\n",
+ " if state is not None and state == c_row * self.n_col + c_col:\n",
+ " image_out[c_row*83:c_row*83+83, c_col*83:c_col*83+83,:] = self.penguin_image\n",
+ "\n",
+ " # Draw the image\n",
+ " plt.imshow(image_out)\n",
+ " self.ax.get_xaxis().set_visible(False)\n",
+ " self.ax.get_yaxis().set_visible(False)\n",
+ " self.ax.spines['top'].set_visible(False)\n",
+ " self.ax.spines['right'].set_visible(False)\n",
+ " self.ax.spines['bottom'].set_visible(False)\n",
+ " self.ax.spines['left'].set_visible(False)\n",
+ "\n",
+ " if draw_state_index:\n",
+ " for c_cell in range(layout.size):\n",
+ " self.draw_text(\"%d\"%(c_cell), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'tl','k')\n",
+ "\n",
+ " # Draw the policy as triangles\n",
+ " if policy is not None:\n",
+ " # If the policy is deterministic\n",
+ " if len(policy) == len(layout):\n",
+ " for i in range(len(layout)):\n",
+ " self.draw_deterministic_policy(i, policy[i])\n",
+ " # Else it is stochastic\n",
+ " else:\n",
+ " for i in range(len(layout)):\n",
+ " self.draw_stochastic_policy(i,policy[:,i])\n",
+ "\n",
+ "\n",
+ " if path1 is not None:\n",
+ " self.draw_path(path1, np.array([1.0, 0.0, 0.0]), np.array([0.0, 1.0, 1.0]))\n",
+ "\n",
+ " if rewards is not None:\n",
+ " for c_cell in range(layout.size):\n",
+ " self.draw_text(\"%d\"%(rewards[c_cell]), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'tr','r')\n",
+ "\n",
+ " if state_values is not None:\n",
+ " for c_cell in range(layout.size):\n",
+ " self.draw_text(\"%2.2f\"%(state_values[c_cell]), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'bc','hotpink')\n",
+ "\n",
+ "\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "Gq1HfJsHN3SB"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# We're going to work on the problem depicted in figure 19.10a\n",
+ "n_rows = 4; n_cols = 4\n",
+ "layout = np.zeros(n_rows * n_cols)\n",
+ "rewards = np.zeros(n_rows * n_cols)\n",
+ "layout[9] = 1 ; rewards[9] = -2\n",
+ "layout[10] = 1; rewards[10] = -2\n",
+ "layout[14] = 1; rewards[14] = -2\n",
+ "layout[15] = 2; rewards[15] = 3\n",
+ "initial_state = 0\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, state = initial_state, rewards=rewards, draw_state_index = True)"
+ ],
+ "metadata": {
+ "id": "eBQ7lTpJQBSe"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "For clarity, the black numbers are the state number and the red numbers are the reward for being in that state. Note that the states are indexed from 0 rather than 1 as in the book to make the code neater."
+ ],
+ "metadata": {
+ "id": "6Vku6v_se2IG"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define a step from the Markov process"
+ ],
+ "metadata": {
+ "id": "axllRDDuDDLS"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's define the state transition function $Pr(s_{t+1}|s_{t},a)$ in full where $a$ is the actions. Here $a=0$ means try to go upward, $a=1$, right, $a=2$ down and $a=3$ right. However, the ice is slippery, so we don't always go the direction we want to.\n",
+ "\n",
+ "Note that as for the states, we've indexed the actions from zero (unlike in the book) so they map to the indices of arrays better"
+ ],
+ "metadata": {
+ "id": "Fhc6DzZNOjiC"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "transition_probabilities_given_action0 = np.array(\\\n",
+ "[[0.00 , 0.33, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.50 , 0.00, 0.33, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.33, 0.00, 0.50, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.33, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.50 , 0.00, 0.00, 0.00, 0.00, 0.17, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.34, 0.00, 0.00, 0.25, 0.00, 0.17, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.34, 0.00, 0.00, 0.17, 0.00, 0.25, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.50, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.17, 0.00, 0.00, 0.75, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.25, 0.00, 0.17, 0.00, 0.00, 0.50, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.17, 0.00, 0.25, 0.00, 0.00, 0.50, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.75 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.25, 0.00, 0.25, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.25, 0.00, 0.25 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.25, 0.00 ],\n",
+ "])\n",
+ "\n",
+ "transition_probabilities_given_action1 = np.array(\\\n",
+ "[[0.00 , 0.25, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.75 , 0.00, 0.25, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.50, 0.00, 0.50, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.33, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.25 , 0.00, 0.00, 0.00, 0.00, 0.17, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.25, 0.00, 0.00, 0.50, 0.00, 0.17, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.25, 0.00, 0.00, 0.50, 0.00, 0.33, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.50, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.33, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.17, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.50, 0.00, 0.17, 0.00, 0.00, 0.25, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.50, 0.00, 0.33, 0.00, 0.00, 0.25, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.34, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.50 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.75, 0.00, 0.25, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.50, 0.00, 0.50 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.34, 0.00, 0.00, 0.50, 0.00 ],\n",
+ "])\n",
+ "\n",
+ "transition_probabilities_given_action2 = np.array(\\\n",
+ "[[0.00 , 0.25, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.25 , 0.00, 0.25, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.25, 0.00, 0.25, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.75 , 0.00, 0.00, 0.00, 0.00, 0.17, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.50, 0.00, 0.00, 0.25, 0.00, 0.17, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.50, 0.00, 0.00, 0.16, 0.00, 0.25, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.75, 0.00, 0.00, 0.16, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.17, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.25, 0.00, 0.17, 0.00, 0.00, 0.33, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.16, 0.00, 0.25, 0.00, 0.00, 0.33, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.16, 0.00, 0.00, 0.00, 0.00, 0.50 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.33, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.50, 0.00, 0.33, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.34, 0.00, 0.50 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.34, 0.00 ],\n",
+ "])\n",
+ "\n",
+ "transition_probabilities_given_action3 = np.array(\\\n",
+ "[[0.00 , 0.25, 0.00, 0.00, 0.33, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.50 , 0.00, 0.25, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.50, 0.00, 0.75, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.50, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.50 , 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.33, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.25, 0.00, 0.00, 0.33, 0.00, 0.50, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.25, 0.00, 0.00, 0.17, 0.00, 0.50, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.25, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.34, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00, 0.50, 0.00, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.33, 0.00, 0.50, 0.00, 0.00, 0.25, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.17, 0.00, 0.50, 0.00, 0.00, 0.25, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.17, 0.00, 0.00, 0.00, 0.00, 0.25 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.34, 0.00, 0.00, 0.00, 0.00, 0.50, 0.00, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.50, 0.00, 0.50, 0.00 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.16, 0.00, 0.00, 0.25, 0.00, 0.75 ],\n",
+ " [0.00 , 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.25, 0.00, 0.00, 0.25, 0.00 ],\n",
+ "])\n",
+ "\n",
+ "# Store all of these in a three dimension array\n",
+ "# Pr(s_{t+1}=2|s_{t}=1, a_{t}=3] is stored at position [2,1,3]\n",
+ "transition_probabilities_given_action = np.concatenate((np.expand_dims(transition_probabilities_given_action0,2),\n",
+ " np.expand_dims(transition_probabilities_given_action1,2),\n",
+ " np.expand_dims(transition_probabilities_given_action2,2),\n",
+ " np.expand_dims(transition_probabilities_given_action3,2)),axis=2)"
+ ],
+ "metadata": {
+ "id": "l7rT78BbOgTi"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Update the state values for the current policy, by making the values at at adjacent\n",
+ "# states compatible with the Bellman equation (equation 19.11)\n",
+ "def policy_evaluation(policy, state_values, rewards, transition_probabilities_given_action, gamma):\n",
+ "\n",
+ " n_state = len(state_values)\n",
+ " state_values_new = np.zeros_like(state_values)\n",
+ "\n",
+ " for state in range(n_state):\n",
+ " # Special case -- bottom right is terminating state, always just rewards 3.0\n",
+ " if state == 15:\n",
+ " state_values_new[state] = 3.0\n",
+ " break\n",
+ "\n",
+ " return state_values_new\n",
+ "\n",
+ "# Greedily choose the action that maximizes the value for each state.\n",
+ "def policy_improvement(state_values, rewards, transition_probabilities_given_action, gamma):\n",
+ " policy = np.zeros_like(state_values, dtype='uint8')\n",
+ " for state in range(15):\n",
+ " # TODO -- Write this function (from equation 19.12)\n",
+ " # Replace this line\n",
+ " policy[state] = 1\n",
+ "\n",
+ "\n",
+ " return policy\n",
+ "\n",
+ "\n",
+ "# Main routine -- alternately call policy evaluation and policy improvement\n",
+ "def dynamic_programming(policy, state_values, rewards, transition_probabilities_given_action, gamma, n_iter, verbose = False):\n",
+ "\n",
+ " for c_iter in range(n_iter):\n",
+ " print(\"Iteration %d\"%(c_iter))\n",
+ "\n",
+ " state_values = policy_evaluation(policy, state_values, rewards, transition_probabilities_given_action, gamma)\n",
+ "\n",
+ " if verbose:\n",
+ " print(\"Updated state values\")\n",
+ " print(\"Policy: \", policy)\n",
+ " print(\"State values:\", state_values)\n",
+ " mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ " mdp_drawer.draw(layout, policy = policy, state_values=state_values)\n",
+ "\n",
+ " policy = policy_improvement(state_values, rewards, transition_probabilities_given_action, gamma)\n",
+ "\n",
+ " if verbose:\n",
+ " print(\"Updated policy values\")\n",
+ " print(\"Policy:\", policy)\n",
+ " print(\"State_values\", state_values)\n",
+ " mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ " mdp_drawer.draw(layout, policy = policy, state_values=state_values)\n",
+ "\n",
+ " return policy, state_values\n"
+ ],
+ "metadata": {
+ "id": "bFYvF9nAloIA"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set seed so random numbers always the same\n",
+ "np.random.seed(0)\n",
+ "\n",
+ "# Let's start with by setting the policy randomly\n",
+ "policy = np.random.choice(size= n_rows * n_cols, a=np.arange(0,4,1))\n",
+ "state_values = np.zeros(n_rows* n_cols)\n",
+ "\n",
+ "# Let's draw the policy first\n",
+ "print(\"Initial state\")\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, policy = policy, rewards = rewards, state_values=state_values, draw_state_index = True)\n",
+ "\n",
+ "n_iter = 2\n",
+ "gamma = 0.9\n",
+ "policy, state_values = dynamic_programming(policy, state_values, rewards, transition_probabilities_given_action, gamma, n_iter, verbose=True)\n",
+ "\n",
+ "print(\"Your state values=\", state_values)\n",
+ "print(\"True values= [ 0. 0. 0. 0. 0. -0.288 -0.288 0. -0.45 -2.288 -2.594 0.9 0. -0.9 -1.1 3. ] \", )"
+ ],
+ "metadata": {
+ "id": "8jWhDlkaKj7Q"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's run it for a series of iterations without drawing."
+ ],
+ "metadata": {
+ "id": "wdcecFKlx97N"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Set seed so random numbers always the same\n",
+ "np.random.seed(0)\n",
+ "\n",
+ "# Let's start with by setting the policy randomly\n",
+ "policy = np.random.choice(size= n_rows * n_cols, a=np.arange(0,4,1))\n",
+ "state_values = np.zeros(n_rows* n_cols)\n",
+ "\n",
+ "# Let's draw the policy first\n",
+ "print(\"Initial state\")\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, policy = policy, rewards = rewards, state_values=state_values, draw_state_index = True)\n",
+ "\n",
+ "n_iter = 20\n",
+ "gamma = 0.9\n",
+ "policy, state_values = dynamic_programming(policy, state_values, rewards, transition_probabilities_given_action, gamma, n_iter, verbose=False)\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, policy = policy, rewards = rewards, state_values=state_values, draw_state_index = True)\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "rtsLUwi6ZEWL"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "You should see that if we start at state 13, the actions have been selected to go all the way around the holes in the ice (keeping a wide berth to avoid slipping into them) and eventually converge on the fish."
+ ],
+ "metadata": {
+ "id": "tvXOs9VhyWnh"
+ }
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap19/19_3_Monte_Carlo_Methods.ipynb b/Notebooks/Chap19/19_3_Monte_Carlo_Methods.ipynb
new file mode 100644
index 0000000..0f42d3d
--- /dev/null
+++ b/Notebooks/Chap19/19_3_Monte_Carlo_Methods.ipynb
@@ -0,0 +1,749 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ },
+ "source": [
+ "# **Notebook 19.3: Monte-Carlo methods**\n",
+ "\n",
+ "This notebook investigates Monte Carlo methods for tabular reinforcement learning as described in section 19.3.2 of the book\n",
+ "\n",
+ "NOTE! There is a mistake in Figure 19.11 in the first printing of the book, so check the errata to avoid becoming confused. Apologies!\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
+ "\n",
+ "Thanks to [Akshil Patel](https://www.akshilpatel.com) and [Jessica Nicholson](https://jessicanicholson1.github.io) for their help in preparing this notebook."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from PIL import Image\n",
+ "\n",
+ "from IPython.display import clear_output\n",
+ "from time import sleep"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "ZsvrUszPLyEG"
+ },
+ "outputs": [],
+ "source": [
+ "# Get local copies of components of images\n",
+ "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Empty.png\n",
+ "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Hole.png\n",
+ "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Fish.png\n",
+ "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Penguin.png"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Gq1HfJsHN3SB"
+ },
+ "outputs": [],
+ "source": [
+ "# Ugly class that takes care of drawing pictures like in the book.\n",
+ "# You can totally ignore this code!\n",
+ "class DrawMDP:\n",
+ " # Constructor initializes parameters\n",
+ " def __init__(self, n_row, n_col):\n",
+ " self.empty_image = np.asarray(Image.open('Empty.png'))\n",
+ " self.hole_image = np.asarray(Image.open('Hole.png'))\n",
+ " self.fish_image = np.asarray(Image.open('Fish.png'))\n",
+ " self.penguin_image = np.asarray(Image.open('Penguin.png'))\n",
+ " self.fig,self.ax = plt.subplots()\n",
+ " self.n_row = n_row\n",
+ " self.n_col = n_col\n",
+ "\n",
+ " my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ " my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ " r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ " g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ " b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ " self.colormap = np.vstack((r,g,b)).transpose()/255.0\n",
+ "\n",
+ "\n",
+ " def draw_text(self, text, row, col, position, color):\n",
+ " if position == 'bc':\n",
+ " self.ax.text( 83*col+41,83 * (row+1) -5, text, horizontalalignment=\"center\", color=color, fontweight='bold')\n",
+ " if position == 'tc':\n",
+ " self.ax.text( 83*col+41,83 * (row) +10, text, horizontalalignment=\"center\", color=color, fontweight='bold')\n",
+ " if position == 'lc':\n",
+ " self.ax.text( 83*col+2,83 * (row) +41, text, verticalalignment=\"center\", color=color, fontweight='bold', rotation=90)\n",
+ " if position == 'rc':\n",
+ " self.ax.text( 83*(col+1)-5,83 * (row) +41, text, horizontalalignment=\"right\", verticalalignment=\"center\", color=color, fontweight='bold', rotation=-90)\n",
+ " if position == 'tl':\n",
+ " self.ax.text( 83*col+5,83 * row +5, text, verticalalignment = 'top', horizontalalignment=\"left\", color=color, fontweight='bold')\n",
+ " if position == 'tr':\n",
+ " self.ax.text( 83*(col+1)-5, 83 * row +5, text, verticalalignment = 'top', horizontalalignment=\"right\", color=color, fontweight='bold')\n",
+ "\n",
+ " # Draws a set of states\n",
+ " def draw_path(self, path, color1, color2):\n",
+ " for i in range(len(path)-1):\n",
+ " row_start = np.floor(path[i]/self.n_col)\n",
+ " row_end = np.floor(path[i+1]/self.n_col)\n",
+ " col_start = path[i] - row_start * self.n_col\n",
+ " col_end = path[i+1] - row_end * self.n_col\n",
+ "\n",
+ " color_index = int(np.floor(255 * i/(len(path)-1.)))\n",
+ " self.ax.plot([col_start * 83+41 + i, col_end * 83+41 + i ],[row_start * 83+41 + i, row_end * 83+41 + i ], color=(self.colormap[color_index,0],self.colormap[color_index,1],self.colormap[color_index,2]))\n",
+ "\n",
+ "\n",
+ " # Draw deterministic policy\n",
+ " def draw_deterministic_policy(self,i, action):\n",
+ " row = np.floor(i/self.n_col)\n",
+ " col = i - row * self.n_col\n",
+ " center_x = 83 * col + 41\n",
+ " center_y = 83 * row + 41\n",
+ " arrow_base_width = 10\n",
+ " arrow_height = 15\n",
+ " # Draw arrow pointing upward\n",
+ " if action ==0:\n",
+ " triangle_indices = np.array([[center_x, center_y-arrow_height/2],\n",
+ " [center_x - arrow_base_width/2, center_y+arrow_height/2],\n",
+ " [center_x + arrow_base_width/2, center_y+arrow_height/2]])\n",
+ " # Draw arrow pointing right\n",
+ " if action ==1:\n",
+ " triangle_indices = np.array([[center_x + arrow_height/2, center_y],\n",
+ " [center_x - arrow_height/2, center_y-arrow_base_width/2],\n",
+ " [center_x - arrow_height/2, center_y+arrow_base_width/2]])\n",
+ " # Draw arrow pointing downward\n",
+ " if action ==2:\n",
+ " triangle_indices = np.array([[center_x, center_y+arrow_height/2],\n",
+ " [center_x - arrow_base_width/2, center_y-arrow_height/2],\n",
+ " [center_x + arrow_base_width/2, center_y-arrow_height/2]])\n",
+ " # Draw arrow pointing left\n",
+ " if action ==3:\n",
+ " triangle_indices = np.array([[center_x - arrow_height/2, center_y],\n",
+ " [center_x + arrow_height/2, center_y-arrow_base_width/2],\n",
+ " [center_x + arrow_height/2, center_y+arrow_base_width/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ " # Draw stochastic policy\n",
+ " def draw_stochastic_policy(self,i, action_probs):\n",
+ " row = np.floor(i/self.n_col)\n",
+ " col = i - row * self.n_col\n",
+ " offset = 20\n",
+ " # Draw arrow pointing upward\n",
+ " center_x = 83 * col + 41\n",
+ " center_y = 83 * row + 41 - offset\n",
+ " arrow_base_width = 15 * action_probs[0]\n",
+ " arrow_height = 20 * action_probs[0]\n",
+ " triangle_indices = np.array([[center_x, center_y-arrow_height/2],\n",
+ " [center_x - arrow_base_width/2, center_y+arrow_height/2],\n",
+ " [center_x + arrow_base_width/2, center_y+arrow_height/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ " # Draw arrow pointing right\n",
+ " center_x = 83 * col + 41 + offset\n",
+ " center_y = 83 * row + 41\n",
+ " arrow_base_width = 15 * action_probs[1]\n",
+ " arrow_height = 20 * action_probs[1]\n",
+ " triangle_indices = np.array([[center_x + arrow_height/2, center_y],\n",
+ " [center_x - arrow_height/2, center_y-arrow_base_width/2],\n",
+ " [center_x - arrow_height/2, center_y+arrow_base_width/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ " # Draw arrow pointing downward\n",
+ " center_x = 83 * col + 41\n",
+ " center_y = 83 * row + 41 +offset\n",
+ " arrow_base_width = 15 * action_probs[2]\n",
+ " arrow_height = 20 * action_probs[2]\n",
+ " triangle_indices = np.array([[center_x, center_y+arrow_height/2],\n",
+ " [center_x - arrow_base_width/2, center_y-arrow_height/2],\n",
+ " [center_x + arrow_base_width/2, center_y-arrow_height/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ " # Draw arrow pointing left\n",
+ " center_x = 83 * col + 41 -offset\n",
+ " center_y = 83 * row + 41\n",
+ " arrow_base_width = 15 * action_probs[3]\n",
+ " arrow_height = 20 * action_probs[3]\n",
+ " triangle_indices = np.array([[center_x - arrow_height/2, center_y],\n",
+ " [center_x + arrow_height/2, center_y-arrow_base_width/2],\n",
+ " [center_x + arrow_height/2, center_y+arrow_base_width/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ "\n",
+ " def draw(self, layout, state=None, draw_state_index= False, rewards=None, policy=None, state_values=None, state_action_values=None,path1=None, path2 = None):\n",
+ " # Construct the image\n",
+ " image_out = np.zeros((self.n_row * 83, self.n_col * 83, 4),dtype='uint8')\n",
+ " for c_row in range (self.n_row):\n",
+ " for c_col in range(self.n_col):\n",
+ " if layout[c_row * self.n_col + c_col]==0:\n",
+ " image_out[c_row*83:c_row*83+83, c_col*83:c_col*83+83,:] = self.empty_image\n",
+ " elif layout[c_row * self.n_col + c_col]==1:\n",
+ " image_out[c_row*83:c_row*83+83, c_col*83:c_col*83+83,:] = self.hole_image\n",
+ " else:\n",
+ " image_out[c_row*83:c_row*83+83, c_col*83:c_col*83+83,:] = self.fish_image\n",
+ " if state is not None and state == c_row * self.n_col + c_col:\n",
+ " image_out[c_row*83:c_row*83+83, c_col*83:c_col*83+83,:] = self.penguin_image\n",
+ "\n",
+ " # Draw the image\n",
+ " plt.imshow(image_out)\n",
+ " self.ax.get_xaxis().set_visible(False)\n",
+ " self.ax.get_yaxis().set_visible(False)\n",
+ " self.ax.spines['top'].set_visible(False)\n",
+ " self.ax.spines['right'].set_visible(False)\n",
+ " self.ax.spines['bottom'].set_visible(False)\n",
+ " self.ax.spines['left'].set_visible(False)\n",
+ "\n",
+ " if draw_state_index:\n",
+ " for c_cell in range(layout.size):\n",
+ " self.draw_text(\"%d\"%(c_cell), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'tl','k')\n",
+ "\n",
+ " # Draw the policy as triangles\n",
+ " if policy is not None:\n",
+ " # If the policy is deterministic\n",
+ " if len(policy) == len(layout):\n",
+ " for i in range(len(layout)):\n",
+ " self.draw_deterministic_policy(i, policy[i])\n",
+ " # Else it is stochastic\n",
+ " else:\n",
+ " for i in range(len(layout)):\n",
+ " self.draw_stochastic_policy(i,policy[:,i])\n",
+ "\n",
+ "\n",
+ " if path1 is not None:\n",
+ " self.draw_path(path1, np.array([1.0, 0.0, 0.0]), np.array([0.0, 1.0, 1.0]))\n",
+ "\n",
+ " if rewards is not None:\n",
+ " for c_cell in range(layout.size):\n",
+ " self.draw_text(\"%d\"%(rewards[c_cell]), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'tr','r')\n",
+ "\n",
+ " if state_values is not None:\n",
+ " for c_cell in range(layout.size):\n",
+ " self.draw_text(\"%2.2f\"%(state_values[c_cell]), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'bc','black')\n",
+ "\n",
+ " if state_action_values is not None:\n",
+ " for c_cell in range(layout.size):\n",
+ " self.draw_text(\"%2.2f\"%(state_action_values[0, c_cell]), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'tc','black')\n",
+ " self.draw_text(\"%2.2f\"%(state_action_values[1, c_cell]), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'rc','black')\n",
+ " self.draw_text(\"%2.2f\"%(state_action_values[2, c_cell]), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'bc','black')\n",
+ " self.draw_text(\"%2.2f\"%(state_action_values[3, c_cell]), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'lc','black')\n",
+ "\n",
+ "\n",
+ "\n",
+ " plt.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "eBQ7lTpJQBSe"
+ },
+ "outputs": [],
+ "source": [
+ "# We're going to work on the problem depicted in figure 19.10a\n",
+ "n_rows = 4; n_cols = 4\n",
+ "layout = np.zeros(n_rows * n_cols)\n",
+ "reward_structure = np.zeros(n_rows * n_cols)\n",
+ "layout[9] = 1 ; reward_structure[9] = -2 # Hole\n",
+ "layout[10] = 1; reward_structure[10] = -2 # Hole\n",
+ "layout[14] = 1; reward_structure[14] = -2 # Hole\n",
+ "layout[15] = 2; reward_structure[15] = 3 # Fish\n",
+ "initial_state = 0\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, state = initial_state, rewards=reward_structure, draw_state_index = True)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "6Vku6v_se2IG"
+ },
+ "source": [
+ "For clarity, the black numbers are the state number and the red numbers are the reward for being in that state. Note that the states are indexed from 0 rather than 1 as in the book to make the code neater."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Fhc6DzZNOjiC"
+ },
+ "source": [
+ "Now let's define the state transition function $Pr(s_{t+1}|s_{t},a)$ in full where $a$ is the actions. Here $a=0$ means try to go upward, $a=1$, right, $a=2$ down and $a=3$ right. However, the ice is slippery, so we don't always go the direction we want to.\n",
+ "\n",
+ "Note that as for the states, we've indexed the actions from zero (unlike in the book) so they map to the indices of arrays better"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "l7rT78BbOgTi"
+ },
+ "outputs": [],
+ "source": [
+ "transition_probabilities_given_action0 = np.array(\\\n",
+ "[[0.90, 0.05, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.05, 0.85, 0.05, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.05, 0.85, 0.05, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.05, 0.90, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.05, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.10, 0.05, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.05, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00]])\n",
+ "\n",
+ "\n",
+ "transition_probabilities_given_action1 = np.array(\\\n",
+ "[[0.10, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.85, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.85, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.85, 0.90, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.05, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.85, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.85, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.10, 0.05, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.05, 0.05, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.05, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00]])\n",
+ "\n",
+ "\n",
+ "transition_probabilities_given_action2 = np.array(\\\n",
+ "[[0.10, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.05, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.05, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.05, 0.10, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.85, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.90, 0.05, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.85, 0.05, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.85, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00]])\n",
+ "\n",
+ "transition_probabilities_given_action3 = np.array(\\\n",
+ "[[0.90, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.05, 0.05, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.05, 0.05, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.05, 0.10, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.05, 0.00, 0.00, 0.00, 0.85, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.85, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.90, 0.85, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.85, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00]])\n",
+ "\n",
+ "\n",
+ "\n",
+ "# Store all of these in a three dimension array\n",
+ "# Pr(s_{t+1}=2|s_{t}=1, a_{t}=3] is stored at position [2,1,3]\n",
+ "transition_probabilities_given_action = np.concatenate((np.expand_dims(transition_probabilities_given_action0,2),\n",
+ " np.expand_dims(transition_probabilities_given_action1,2),\n",
+ " np.expand_dims(transition_probabilities_given_action2,2),\n",
+ " np.expand_dims(transition_probabilities_given_action3,2)),axis=2)\n",
+ "\n",
+ "print('Grid Size:', len(transition_probabilities_given_action[0]))\n",
+ "print()\n",
+ "print('Transition Probabilities for when next state = 2:')\n",
+ "print(transition_probabilities_given_action[2])\n",
+ "print()\n",
+ "print('Transitions Probabilities for when next state = 2 and current state = 1')\n",
+ "print(transition_probabilities_given_action[2][1])\n",
+ "print()\n",
+ "print('Transitions Probabilities for when next state = 2 and current state = 1 and action = 3 (Left):')\n",
+ "print(transition_probabilities_given_action[2][1][3])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "BHWjp6Qq4tBF"
+ },
+ "source": [
+ "## Implementation Details\n",
+ "\n",
+ "We provide the following methods:\n",
+ "\n",
+ "- **`markov_decision_process_step_stochastic`** - this function selects an action based on the stochastic policy for the current state, updates the state based on the transition probabilities associated with the chosen action, and returns the new state, the reward obtained for the new state, the chosen action, and whether the episode terminates.\n",
+ "\n",
+ "- **`get_one_episode`** - this function simulates an episode of agent-environment interaction. It returns the states, rewards, and actions seen in that episode, which we can then use to update the agent.\n",
+ "\n",
+ "- **`calculate_returns`** - this function calls on the **`calculate_return`** function that computes the discounted sum of rewards from a specific step, in a sequence of rewards.\n",
+ "\n",
+ "You have to implement the following methods:\n",
+ "\n",
+ "- **`deterministic_policy_to_epsilon_greedy`** - given a deterministic policy, where one action is chosen per state, this function computes the $\\epsilon$-greedy version of that policy, where each of the four actions has some nonzero probability of being selected per state. In each state, the probability of selecting each of the actions should sum to 1.\n",
+ "\n",
+ "- **`update_policy_mc`** - this function updates the action-value function using the Monte Carlo method. We use the rollout trajectories collected using `get_one_episode` to calculate the returns. Then update the action values towards the Monte Carlo estimate of the return for each state."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "akjrncMF-FkU"
+ },
+ "outputs": [],
+ "source": [
+ "# This takes a single step from an MDP\n",
+ "def markov_decision_process_step_stochastic(state, transition_probabilities_given_action, reward_structure, terminal_states, stochastic_policy):\n",
+ " # Pick action\n",
+ " action = np.random.choice(a=np.arange(0,4,1),p=stochastic_policy[:,state])\n",
+ "\n",
+ " # Update the state\n",
+ " new_state = np.random.choice(a=np.arange(0,transition_probabilities_given_action.shape[0]),p = transition_probabilities_given_action[:,state,action])\n",
+ " # Return the reward\n",
+ " reward = reward_structure[new_state]\n",
+ " is_terminal = new_state in [terminal_states]\n",
+ "\n",
+ " return new_state, reward, action, is_terminal"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "bFYvF9nAloIA"
+ },
+ "outputs": [],
+ "source": [
+ "# Run one episode and return actions, rewards, returns\n",
+ "def get_one_episode(initial_state, transition_probabilities_given_action, reward_structure, terminal_states, stochastic_policy):\n",
+ "\n",
+ " states = []\n",
+ " rewards = []\n",
+ " actions = []\n",
+ "\n",
+ " states.append(initial_state)\n",
+ " state = initial_state\n",
+ "\n",
+ " is_terminal = False\n",
+ " # While we haven't reached a terminal state\n",
+ " while not is_terminal:\n",
+ " # Keep stepping through MDP\n",
+ " state, reward, action, is_terminal = markov_decision_process_step_stochastic(state,\n",
+ " transition_probabilities_given_action,\n",
+ " reward_structure,\n",
+ " terminal_states,\n",
+ " stochastic_policy)\n",
+ " states.append(state)\n",
+ " rewards.append(reward)\n",
+ " actions.append(action)\n",
+ "\n",
+ " states = np.array(states, dtype=\"uint8\")\n",
+ " rewards = np.array(rewards)\n",
+ " actions = np.array(actions, dtype=\"uint8\")\n",
+ "\n",
+ " # If the episode was terminated early, we need to compute the return differently using r_{t+1} + gamma*V(s_{t+1})\n",
+ " return states, rewards, actions"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "qJhOrIId4tBF"
+ },
+ "outputs": [],
+ "source": [
+ "def visualize_one_episode(states, actions):\n",
+ " # Define actions for visualization\n",
+ " acts = ['up', 'right', 'down', 'left']\n",
+ "\n",
+ " # Iterate over the states and actions\n",
+ " for i in range(len(states)):\n",
+ "\n",
+ " if i == 0:\n",
+ " print('Starting State:', states[i])\n",
+ "\n",
+ " elif i == len(states)-1:\n",
+ " print('Episode Done:', states[i])\n",
+ "\n",
+ " else:\n",
+ " print('State', states[i-1])\n",
+ " a = actions[i]\n",
+ " print('Action:', acts[a])\n",
+ " print('Next State:', states[i])\n",
+ "\n",
+ " # Visualize the current state using the MDP drawer\n",
+ " mdp_drawer.draw(layout, state=states[i], rewards=reward_structure, draw_state_index=True)\n",
+ " clear_output(True)\n",
+ "\n",
+ " # Pause for a short duration to allow observation\n",
+ " sleep(1.5)\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "_AKwdtQQHzIK"
+ },
+ "outputs": [],
+ "source": [
+ "# Convert deterministic policy (1x16) to an epsilon greedy stochastic policy (4x16)\n",
+ "def deterministic_policy_to_epsilon_greedy(policy, epsilon=0.2):\n",
+ " # TODO -- write this function\n",
+ " # You should wind up with a 4x16 matrix, with epsilon/3 in every position except the real policy\n",
+ " # The columns should sum to one\n",
+ " # Replace this line:\n",
+ " stochastic_policy = np.ones((4,16)) * 0.25\n",
+ "\n",
+ "\n",
+ " return stochastic_policy"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "OhVXw2Favo-w"
+ },
+ "source": [
+ "Let's try generating an episode"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "5zQ1Oh9Zvnwt"
+ },
+ "outputs": [],
+ "source": [
+ "# Set seed so random numbers always the same\n",
+ "np.random.seed(6)\n",
+ "# Print in compact form\n",
+ "np.set_printoptions(precision=3)\n",
+ "\n",
+ "# Let's start with by setting the policy randomly\n",
+ "policy = np.random.choice(size= n_rows * n_cols, a=np.arange(0,4,1))\n",
+ "\n",
+ "# Convert deterministic policy to stochastic\n",
+ "stochastic_policy = deterministic_policy_to_epsilon_greedy(policy)\n",
+ "\n",
+ "print(\"Initial Penguin Policy:\")\n",
+ "print(policy)\n",
+ "print()\n",
+ "print('Stochastic Penguin Policy:')\n",
+ "print(stochastic_policy)\n",
+ "print()\n",
+ "\n",
+ "initial_state = 5\n",
+ "terminal_states=[15]\n",
+ "states, rewards, actions = get_one_episode(initial_state,transition_probabilities_given_action, reward_structure, terminal_states, stochastic_policy)\n",
+ "\n",
+ "print('Initial Penguin Position:')\n",
+ "mdp_drawer.draw(layout, state = initial_state, rewards=reward_structure, draw_state_index = True)\n",
+ "\n",
+ "print('Total steps to termination:', len(states))\n",
+ "print('Final Reward:', np.sum(rewards))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "KJH-UGKk4tBF"
+ },
+ "outputs": [],
+ "source": [
+ "#this visualizes the complete episode\n",
+ "visualize_one_episode(states, actions)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "nl6rtNffwhcU"
+ },
+ "source": [
+ "We'll need to calculate the returns (discounted cumulative reward) for each state action pair"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "FxrItqGPLTq7"
+ },
+ "outputs": [],
+ "source": [
+ "def calculate_returns(rewards, gamma):\n",
+ " returns = np.zeros(len(rewards))\n",
+ " for c_return in range(len(returns)):\n",
+ " returns[c_return] = calculate_return(rewards[c_return:], gamma)\n",
+ " return returns\n",
+ "\n",
+ "def calculate_return(rewards, gamma):\n",
+ " return_val = 0.0\n",
+ " for i in range(len(rewards)):\n",
+ " return_val += rewards[i] * np.power(gamma, i)\n",
+ " return return_val"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "DX1KfHRhzUOU"
+ },
+ "source": [
+ "This routine does the main work of the on-policy Monte Carlo method. We repeatedly rollout episods, calculate the returns. Then we figure out the average return for each state action pair, and choose the next policy as the action that has greatest state action value at each state."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "hCghcKlOJXSM"
+ },
+ "outputs": [],
+ "source": [
+ "def update_policy_mc(initial_state, transition_probabilities_given_action, reward_structure, terminal_states, stochastic_policy, gamma, n_rollouts=1):\n",
+ " # Create two matrices to store total returns for each action/state pair and the\n",
+ " # number of times we observed that action/state pair\n",
+ " n_state = transition_probabilities_given_action.shape[0]\n",
+ " n_action = transition_probabilities_given_action.shape[2]\n",
+ " # Contains the total returns seen for taking this action at this state\n",
+ " state_action_returns_total = np.zeros((n_action, n_state))\n",
+ " # Contains the number of times we have taken this action in this state\n",
+ " state_action_count = np.zeros((n_action,n_state))\n",
+ "\n",
+ " # For each rollout\n",
+ " for c_rollout in range(n_rollouts):\n",
+ " # TODO -- Complete this function\n",
+ " # 1. Draw a random state from 0 to 14\n",
+ " # 2. Get one episode starting at that state\n",
+ " # 3. Compute the returns\n",
+ " # 4. For each position in the trajectory, update state_action_returns_total and state_action_count\n",
+ " # Replace these two lines\n",
+ " state_action_returns_total[0,1] = state_action_returns_total[0,1]\n",
+ " state_action_count[0,1] = state_action_count[0,1]\n",
+ "\n",
+ "\n",
+ " # Normalize -- add small number to denominator to avoid divide by zero\n",
+ " state_action_values = state_action_returns_total/( state_action_count+0.00001)\n",
+ " policy = np.argmax(state_action_values, axis=0).astype(int)\n",
+ " return policy, state_action_values\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "8jWhDlkaKj7Q"
+ },
+ "outputs": [],
+ "source": [
+ "# Set seed so random numbers always the same\n",
+ "np.random.seed(0)\n",
+ "# Print in compact form\n",
+ "np.set_printoptions(precision=3)\n",
+ "\n",
+ "# Let's start with by setting the policy randomly\n",
+ "policy = np.random.choice(size= n_rows * n_cols, a=np.arange(0,4,1))\n",
+ "gamma = 0.9\n",
+ "print(\"Initial policy:\")\n",
+ "print(policy)\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, policy = policy, rewards = reward_structure)\n",
+ "\n",
+ "terminal_states = [15]\n",
+ "# Track all the policies so we can visualize them later\n",
+ "all_policies = []\n",
+ "n_policy_update = 2000\n",
+ "for c_policy_update in range(n_policy_update):\n",
+ " # Convert policy to stochastic\n",
+ " stochastic_policy = deterministic_policy_to_epsilon_greedy(policy)\n",
+ " # Update policy by Monte Carlo method\n",
+ " policy, state_action_values = update_policy_mc(initial_state, transition_probabilities_given_action, reward_structure, terminal_states, stochastic_policy, gamma, n_rollouts=100)\n",
+ " all_policies.append(policy)\n",
+ "\n",
+ " # Print out 10 snapshots of progress\n",
+ " if (c_policy_update % (n_policy_update//10) == 0) or c_policy_update == n_policy_update - 1:\n",
+ " print(\"Updated policy\")\n",
+ " print(policy)\n",
+ " mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ " mdp_drawer.draw(layout, policy = policy, rewards = reward_structure, state_action_values=state_action_values)\n",
+ "\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "j7Ny47kTEMzH"
+ },
+ "source": [
+ "You can see a definite improvement to the policy"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.10.12"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
\ No newline at end of file
diff --git a/Notebooks/Chap19/19_4_Temporal_Difference_Methods.ipynb b/Notebooks/Chap19/19_4_Temporal_Difference_Methods.ipynb
new file mode 100644
index 0000000..d85443d
--- /dev/null
+++ b/Notebooks/Chap19/19_4_Temporal_Difference_Methods.ipynb
@@ -0,0 +1,762 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ },
+ "source": [
+ "# **Notebook 19.4: Temporal difference methods**\n",
+ "\n",
+ "This notebook investigates temporal difference methods for tabular reinforcement learning as described in section 19.3.3 of the book\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
+ "\n",
+ "Thanks to [Akshil Patel](https://www.akshilpatel.com) and [Jessica Nicholson](https://jessicanicholson1.github.io) for their help in preparing this notebook."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "from PIL import Image\n",
+ "from IPython.display import clear_output\n",
+ "from time import sleep\n",
+ "from copy import deepcopy"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "ZsvrUszPLyEG"
+ },
+ "outputs": [],
+ "source": [
+ "# Get local copies of components of images\n",
+ "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Empty.png\n",
+ "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Hole.png\n",
+ "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Fish.png\n",
+ "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Penguin.png"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Gq1HfJsHN3SB"
+ },
+ "outputs": [],
+ "source": [
+ "# Ugly class that takes care of drawing pictures like in the book.\n",
+ "# You can totally ignore this code!\n",
+ "class DrawMDP:\n",
+ " # Constructor initializes parameters\n",
+ " def __init__(self, n_row, n_col):\n",
+ " self.empty_image = np.asarray(Image.open('Empty.png'))\n",
+ " self.hole_image = np.asarray(Image.open('Hole.png'))\n",
+ " self.fish_image = np.asarray(Image.open('Fish.png'))\n",
+ " self.penguin_image = np.asarray(Image.open('Penguin.png'))\n",
+ " self.fig,self.ax = plt.subplots()\n",
+ " self.n_row = n_row\n",
+ " self.n_col = n_col\n",
+ "\n",
+ " my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ " my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ " r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ " g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ " b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ " self.colormap = np.vstack((r,g,b)).transpose()/255.0\n",
+ "\n",
+ "\n",
+ " def draw_text(self, text, row, col, position, color):\n",
+ " if position == 'bc':\n",
+ " self.ax.text( 83*col+41,83 * (row+1) -5, text, horizontalalignment=\"center\", color=color, fontweight='bold')\n",
+ " if position == 'tc':\n",
+ " self.ax.text( 83*col+41,83 * (row) +10, text, horizontalalignment=\"center\", color=color, fontweight='bold')\n",
+ " if position == 'lc':\n",
+ " self.ax.text( 83*col+2,83 * (row) +41, text, verticalalignment=\"center\", color=color, fontweight='bold', rotation=90)\n",
+ " if position == 'rc':\n",
+ " self.ax.text( 83*(col+1)-5,83 * (row) +41, text, horizontalalignment=\"right\", verticalalignment=\"center\", color=color, fontweight='bold', rotation=-90)\n",
+ " if position == 'tl':\n",
+ " self.ax.text( 83*col+5,83 * row +5, text, verticalalignment = 'top', horizontalalignment=\"left\", color=color, fontweight='bold')\n",
+ " if position == 'tr':\n",
+ " self.ax.text( 83*(col+1)-5, 83 * row +5, text, verticalalignment = 'top', horizontalalignment=\"right\", color=color, fontweight='bold')\n",
+ "\n",
+ " # Draws a set of states\n",
+ " def draw_path(self, path, color1, color2):\n",
+ " for i in range(len(path)-1):\n",
+ " row_start = np.floor(path[i]/self.n_col)\n",
+ " row_end = np.floor(path[i+1]/self.n_col)\n",
+ " col_start = path[i] - row_start * self.n_col\n",
+ " col_end = path[i+1] - row_end * self.n_col\n",
+ "\n",
+ " color_index = int(np.floor(255 * i/(len(path)-1.)))\n",
+ " self.ax.plot([col_start * 83+41 + i, col_end * 83+41 + i ],[row_start * 83+41 + i, row_end * 83+41 + i ], color=(self.colormap[color_index,0],self.colormap[color_index,1],self.colormap[color_index,2]))\n",
+ "\n",
+ "\n",
+ " # Draw deterministic policy\n",
+ " def draw_deterministic_policy(self,i, action):\n",
+ " row = np.floor(i/self.n_col)\n",
+ " col = i - row * self.n_col\n",
+ " center_x = 83 * col + 41\n",
+ " center_y = 83 * row + 41\n",
+ " arrow_base_width = 10\n",
+ " arrow_height = 15\n",
+ " # Draw arrow pointing upward\n",
+ " if action ==0:\n",
+ " triangle_indices = np.array([[center_x, center_y-arrow_height/2],\n",
+ " [center_x - arrow_base_width/2, center_y+arrow_height/2],\n",
+ " [center_x + arrow_base_width/2, center_y+arrow_height/2]])\n",
+ " # Draw arrow pointing right\n",
+ " if action ==1:\n",
+ " triangle_indices = np.array([[center_x + arrow_height/2, center_y],\n",
+ " [center_x - arrow_height/2, center_y-arrow_base_width/2],\n",
+ " [center_x - arrow_height/2, center_y+arrow_base_width/2]])\n",
+ " # Draw arrow pointing downward\n",
+ " if action ==2:\n",
+ " triangle_indices = np.array([[center_x, center_y+arrow_height/2],\n",
+ " [center_x - arrow_base_width/2, center_y-arrow_height/2],\n",
+ " [center_x + arrow_base_width/2, center_y-arrow_height/2]])\n",
+ " # Draw arrow pointing left\n",
+ " if action ==3:\n",
+ " triangle_indices = np.array([[center_x - arrow_height/2, center_y],\n",
+ " [center_x + arrow_height/2, center_y-arrow_base_width/2],\n",
+ " [center_x + arrow_height/2, center_y+arrow_base_width/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ " # Draw stochastic policy\n",
+ " def draw_stochastic_policy(self,i, action_probs):\n",
+ " row = np.floor(i/self.n_col)\n",
+ " col = i - row * self.n_col\n",
+ " offset = 20\n",
+ " # Draw arrow pointing upward\n",
+ " center_x = 83 * col + 41\n",
+ " center_y = 83 * row + 41 - offset\n",
+ " arrow_base_width = 15 * action_probs[0]\n",
+ " arrow_height = 20 * action_probs[0]\n",
+ " triangle_indices = np.array([[center_x, center_y-arrow_height/2],\n",
+ " [center_x - arrow_base_width/2, center_y+arrow_height/2],\n",
+ " [center_x + arrow_base_width/2, center_y+arrow_height/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ " # Draw arrow pointing right\n",
+ " center_x = 83 * col + 41 + offset\n",
+ " center_y = 83 * row + 41\n",
+ " arrow_base_width = 15 * action_probs[1]\n",
+ " arrow_height = 20 * action_probs[1]\n",
+ " triangle_indices = np.array([[center_x + arrow_height/2, center_y],\n",
+ " [center_x - arrow_height/2, center_y-arrow_base_width/2],\n",
+ " [center_x - arrow_height/2, center_y+arrow_base_width/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ " # Draw arrow pointing downward\n",
+ " center_x = 83 * col + 41\n",
+ " center_y = 83 * row + 41 +offset\n",
+ " arrow_base_width = 15 * action_probs[2]\n",
+ " arrow_height = 20 * action_probs[2]\n",
+ " triangle_indices = np.array([[center_x, center_y+arrow_height/2],\n",
+ " [center_x - arrow_base_width/2, center_y-arrow_height/2],\n",
+ " [center_x + arrow_base_width/2, center_y-arrow_height/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ " # Draw arrow pointing left\n",
+ " center_x = 83 * col + 41 -offset\n",
+ " center_y = 83 * row + 41\n",
+ " arrow_base_width = 15 * action_probs[3]\n",
+ " arrow_height = 20 * action_probs[3]\n",
+ " triangle_indices = np.array([[center_x - arrow_height/2, center_y],\n",
+ " [center_x + arrow_height/2, center_y-arrow_base_width/2],\n",
+ " [center_x + arrow_height/2, center_y+arrow_base_width/2]])\n",
+ " self.ax.fill(triangle_indices[:,0], triangle_indices[:,1],facecolor='cyan', edgecolor='darkcyan', linewidth=1)\n",
+ "\n",
+ "\n",
+ " def draw(self, layout, state=None, draw_state_index= False, rewards=None, policy=None, state_values=None, state_action_values=None,path1=None, path2 = None):\n",
+ " # Construct the image\n",
+ " image_out = np.zeros((self.n_row * 83, self.n_col * 83, 4),dtype='uint8')\n",
+ " for c_row in range (self.n_row):\n",
+ " for c_col in range(self.n_col):\n",
+ " if layout[c_row * self.n_col + c_col]==0:\n",
+ " image_out[c_row*83:c_row*83+83, c_col*83:c_col*83+83,:] = self.empty_image\n",
+ " elif layout[c_row * self.n_col + c_col]==1:\n",
+ " image_out[c_row*83:c_row*83+83, c_col*83:c_col*83+83,:] = self.hole_image\n",
+ " else:\n",
+ " image_out[c_row*83:c_row*83+83, c_col*83:c_col*83+83,:] = self.fish_image\n",
+ " if state is not None and state == c_row * self.n_col + c_col:\n",
+ " image_out[c_row*83:c_row*83+83, c_col*83:c_col*83+83,:] = self.penguin_image\n",
+ "\n",
+ " # Draw the image\n",
+ " plt.imshow(image_out)\n",
+ " self.ax.get_xaxis().set_visible(False)\n",
+ " self.ax.get_yaxis().set_visible(False)\n",
+ " self.ax.spines['top'].set_visible(False)\n",
+ " self.ax.spines['right'].set_visible(False)\n",
+ " self.ax.spines['bottom'].set_visible(False)\n",
+ " self.ax.spines['left'].set_visible(False)\n",
+ "\n",
+ " if draw_state_index:\n",
+ " for c_cell in range(layout.size):\n",
+ " self.draw_text(\"%d\"%(c_cell), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'tl','k')\n",
+ "\n",
+ " # Draw the policy as triangles\n",
+ " if policy is not None:\n",
+ " # If the policy is deterministic\n",
+ " if len(policy) == len(layout):\n",
+ " for i in range(len(layout)):\n",
+ " self.draw_deterministic_policy(i, policy[i])\n",
+ " # Else it is stochastic\n",
+ " else:\n",
+ " for i in range(len(layout)):\n",
+ " self.draw_stochastic_policy(i,policy[:,i])\n",
+ "\n",
+ "\n",
+ " if path1 is not None:\n",
+ " self.draw_path(path1, np.array([1.0, 0.0, 0.0]), np.array([0.0, 1.0, 1.0]))\n",
+ "\n",
+ " if rewards is not None:\n",
+ " for c_cell in range(layout.size):\n",
+ " self.draw_text(\"%d\"%(rewards[c_cell]), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'tr','r')\n",
+ "\n",
+ " if state_values is not None:\n",
+ " for c_cell in range(layout.size):\n",
+ " self.draw_text(\"%2.2f\"%(state_values[c_cell]), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'bc','black')\n",
+ "\n",
+ " if state_action_values is not None:\n",
+ " for c_cell in range(layout.size):\n",
+ " self.draw_text(\"%2.2f\"%(state_action_values[0, c_cell]), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'tc','black')\n",
+ " self.draw_text(\"%2.2f\"%(state_action_values[1, c_cell]), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'rc','black')\n",
+ " self.draw_text(\"%2.2f\"%(state_action_values[2, c_cell]), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'bc','black')\n",
+ " self.draw_text(\"%2.2f\"%(state_action_values[3, c_cell]), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'lc','black')\n",
+ "\n",
+ " plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "JU8gX59o76xM"
+ },
+ "source": [
+ "# Penguin Ice Environment\n",
+ "\n",
+ "In this implementation we have designed an icy gridworld that a penguin has to traverse to reach the fish found in the bottom right corner.\n",
+ "\n",
+ "## Environment Description\n",
+ "\n",
+ "Consider having to cross an icy surface to reach the yummy fish. In order to achieve this task as quickly as possible, the penguin needs to waddle along as fast as it can whilst simultaneously avoiding falling into the holes.\n",
+ "\n",
+ "In this icy environment the penguin is at one of the discrete cells in the gridworld. The agent starts each episode on a randomly chosen cell. The environment state dynamics are captured by the transition probabilities $Pr(s_{t+1} |s_t, a_t)$ where $s_t$ is the current state, $a_t$ is the action chosen, and $s_{t+1}$ is the next state at decision stage t. At each decision stage, the penguin can move in one of four directions: $a=0$ means try to go upward, $a=1$, right, $a=2$ down and $a=3$ left.\n",
+ "\n",
+ "However, the ice is slippery, so we don't always go the direction we want to: every time the agent chooses an action, with 0.25 probability, the environment changes the action taken to a differenct action, which is uniformly sampled from the other available actions.\n",
+ "\n",
+ "The rewards are deterministic; the penguin will receive a reward of +3 if it reaches the fish, -2 if it slips into a hole and 0 otherwise.\n",
+ "\n",
+ "Note that as for the states, we've indexed the actions from zero (unlike in the book) so they map to the indices of arrays better"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "eBQ7lTpJQBSe"
+ },
+ "outputs": [],
+ "source": [
+ "# We're going to work on the problem depicted in figure 19.10a\n",
+ "n_rows = 4; n_cols = 4\n",
+ "layout = np.zeros(n_rows * n_cols)\n",
+ "reward_structure = np.zeros(n_rows * n_cols)\n",
+ "layout[9] = 1 ; reward_structure[9] = -2 # Hole\n",
+ "layout[10] = 1; reward_structure[10] = -2 # Hole\n",
+ "layout[14] = 1; reward_structure[14] = -2 # Hole\n",
+ "layout[15] = 2; reward_structure[15] = 3 # Fish\n",
+ "initial_state = 0\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, state = initial_state, rewards=reward_structure, draw_state_index = True)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "6Vku6v_se2IG"
+ },
+ "source": [
+ "For clarity, the black numbers are the state number and the red numbers are the reward for being in that state. Note that the states are indexed from 0 rather than 1 as in the book to make the code neater."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Fhc6DzZNOjiC"
+ },
+ "source": [
+ "Now let's define the state transition function $Pr(s_{t+1}|s_{t},a)$ in full where $a$ is the actions. Here $a=0$ means try to go upward, $a=1$, right, $a=2$ down and $a=3$ right. However, the ice is slippery, so we don't always go the direction we want to.\n",
+ "\n",
+ "Note that as for the states, we've indexed the actions from zero (unlike in the book) so they map to the indices of arrays better"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "wROjgnqh76xN"
+ },
+ "outputs": [],
+ "source": [
+ "transition_probabilities_given_action0 = np.array(\\\n",
+ "[[0.90, 0.05, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.05, 0.85, 0.05, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.05, 0.85, 0.05, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.05, 0.90, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.05, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.10, 0.05, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.05, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00]])\n",
+ "\n",
+ "\n",
+ "transition_probabilities_given_action1 = np.array(\\\n",
+ "[[0.10, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.85, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.85, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.85, 0.90, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.05, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.85, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.85, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.10, 0.05, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.05, 0.05, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.05, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00]])\n",
+ "\n",
+ "\n",
+ "transition_probabilities_given_action2 = np.array(\\\n",
+ "[[0.10, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.05, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.05, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.05, 0.10, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.85, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.90, 0.05, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.85, 0.05, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.85, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00]])\n",
+ "\n",
+ "transition_probabilities_given_action3 = np.array(\\\n",
+ "[[0.90, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.05, 0.05, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.05, 0.05, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.05, 0.10, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.05, 0.00, 0.00, 0.00, 0.85, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.85, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.90, 0.85, 0.00, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.85, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.00],\n",
+ " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00]])\n",
+ "\n",
+ "\n",
+ "\n",
+ "# Store all of these in a three dimension array\n",
+ "# Pr(s_{t+1}=2|s_{t}=1, a_{t}=3] is stored at position [2,1,3]\n",
+ "transition_probabilities_given_action = np.concatenate((np.expand_dims(transition_probabilities_given_action0,2),\n",
+ " np.expand_dims(transition_probabilities_given_action1,2),\n",
+ " np.expand_dims(transition_probabilities_given_action2,2),\n",
+ " np.expand_dims(transition_probabilities_given_action3,2)),axis=2)\n",
+ "\n",
+ "print('Grid Size:', len(transition_probabilities_given_action[0]))\n",
+ "print()\n",
+ "print('Transition Probabilities for when next state = 2:')\n",
+ "print(transition_probabilities_given_action[2])\n",
+ "print()\n",
+ "print('Transitions Probabilities for when next state = 2 and current state = 1')\n",
+ "print(transition_probabilities_given_action[2][1])\n",
+ "print()\n",
+ "print('Transitions Probabilities for when next state = 2 and current state = 1 and action = 3 (Left):')\n",
+ "print(transition_probabilities_given_action[2][1][3])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "eblSQ6xZ76xN"
+ },
+ "source": [
+ "## Implementation Details\n",
+ "\n",
+ "We provide the following methods:\n",
+ "- **`markov_decision_process_step`** - this function simulates $Pr(s_{t+1} | s_{t}, a_{t})$. It randomly selects an action, updates the state based on the transition probabilities associated with the chosen action, and returns the new state, the reward obtained for leaving the current state, and the chosen action. The randomness in action selection and state transitions reflects a random exploration process and the stochastic nature of the MDP, respectively.\n",
+ "\n",
+ "- **`get_policy`** - this function computes a policy that acts greedily with respect to the state-action values. The policy is computed for all states and the action that maximizes the state-action value is chosen for each state. When there are multiple optimal actions, one is chosen at random.\n",
+ "\n",
+ "\n",
+ "You have to implement the following method:\n",
+ "\n",
+ "- **`q_learning_step`** - this function implements a single step of the Q-learning algorithm for reinforcement learning as shown below. The update follows the Q-learning formula and is controlled by parameters such as the learning rate (alpha) and the discount factor $(\\gamma)$. The function returns the updated state-action values matrix.\n",
+ "\n",
+ "$Q(s, a) \\leftarrow (1 - \\alpha) \\cdot Q(s, a) + \\alpha \\cdot \\left(r + \\gamma \\cdot \\max_{a'} Q(s', a')\\right)$"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "cKLn4Iam76xN"
+ },
+ "outputs": [],
+ "source": [
+ "def get_policy(state_action_values):\n",
+ " policy = np.zeros(state_action_values.shape[1]) # One action for each state\n",
+ " for state in range(state_action_values.shape[1]):\n",
+ " # Break ties for maximising actions randomly\n",
+ " policy[state] = np.random.choice(np.flatnonzero(state_action_values[:, state] == max(state_action_values[:, state])))\n",
+ " return policy"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "akjrncMF-FkU"
+ },
+ "outputs": [],
+ "source": [
+ "def markov_decision_process_step(state, transition_probabilities_given_action, reward_structure, terminal_states, action=None):\n",
+ " # Pick action\n",
+ " if action is None:\n",
+ " action = np.random.randint(4)\n",
+ " # Update the state\n",
+ " new_state = np.random.choice(a=range(transition_probabilities_given_action.shape[0]), p = transition_probabilities_given_action[:, state,action])\n",
+ "\n",
+ " # Return the reward -- here the reward is for arriving at the state\n",
+ " reward = reward_structure[new_state]\n",
+ " is_terminal = new_state in [terminal_states]\n",
+ "\n",
+ " return new_state, reward, action, is_terminal"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "5pO6-9ACWhiV"
+ },
+ "outputs": [],
+ "source": [
+ "def q_learning_step(state_action_values, reward, state, new_state, action, is_terminal, gamma, alpha = 0.1):\n",
+ " # TODO -- write this function\n",
+ " # Replace this line\n",
+ " state_action_values_after = np.copy(state_action_values)\n",
+ "\n",
+ " return state_action_values_after"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "u4OHTTk176xO"
+ },
+ "source": [
+ "Lets run this for a single Q-learning step"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Fu5_VjvbSwfJ"
+ },
+ "outputs": [],
+ "source": [
+ "# Initialize the state-action values to random numbers\n",
+ "np.random.seed(0)\n",
+ "n_state = transition_probabilities_given_action.shape[0]\n",
+ "n_action = transition_probabilities_given_action.shape[2]\n",
+ "terminal_states=[15]\n",
+ "state_action_values = np.random.normal(size=(n_action, n_state))\n",
+ "# Hard code value of termination state of finding fish to 0\n",
+ "state_action_values[:, terminal_states] = 0\n",
+ "gamma = 0.9\n",
+ "\n",
+ "policy = get_policy(state_action_values)\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, policy = policy, state_action_values = state_action_values, rewards = reward_structure)\n",
+ "\n",
+ "# Now let's simulate a single Q-learning step\n",
+ "initial_state = 9\n",
+ "print(\"Initial state =\",initial_state)\n",
+ "new_state, reward, action, is_terminal = markov_decision_process_step(initial_state, transition_probabilities_given_action, reward_structure, terminal_states)\n",
+ "print(\"Action =\",action)\n",
+ "print(\"New state =\",new_state)\n",
+ "print(\"Reward =\", reward)\n",
+ "\n",
+ "state_action_values_after = q_learning_step(state_action_values, reward, initial_state, new_state, action, is_terminal, gamma)\n",
+ "print(\"Your value:\",state_action_values_after[action, initial_state])\n",
+ "print(\"True value: 0.3024718977397814\")\n",
+ "\n",
+ "policy = get_policy(state_action_values)\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, policy = policy, state_action_values = state_action_values_after, rewards = reward_structure)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Ogh0qucmb68J"
+ },
+ "source": [
+ "Now let's run this for a while (20000) steps and watch the policy improve"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "N6gFYifh76xO"
+ },
+ "outputs": [],
+ "source": [
+ "# Initialize the state-action values to random numbers\n",
+ "np.random.seed(0)\n",
+ "n_state = transition_probabilities_given_action.shape[0]\n",
+ "n_action = transition_probabilities_given_action.shape[2]\n",
+ "state_action_values = np.random.normal(size=(n_action, n_state))\n",
+ "\n",
+ "# Hard code value of termination state of finding fish to 0\n",
+ "terminal_states = [15]\n",
+ "state_action_values[:, terminal_states] = 0\n",
+ "gamma = 0.9\n",
+ "\n",
+ "# Draw the initial setup\n",
+ "print('Initial Policy:')\n",
+ "policy = get_policy(state_action_values)\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, policy = policy, state_action_values = state_action_values, rewards = reward_structure)\n",
+ "\n",
+ "state = np.random.randint(n_state-1)\n",
+ "\n",
+ "# Run for a number of iterations\n",
+ "for c_iter in range(20000):\n",
+ " new_state, reward, action, is_terminal = markov_decision_process_step(state, transition_probabilities_given_action, reward_structure, terminal_states)\n",
+ " state_action_values_after = q_learning_step(state_action_values, reward, state, new_state, action, is_terminal, gamma)\n",
+ "\n",
+ " # If in termination state, reset state randomly\n",
+ " if is_terminal:\n",
+ " state = np.random.randint(n_state-1)\n",
+ " else:\n",
+ " state = new_state\n",
+ "\n",
+ " # Update the policy\n",
+ " state_action_values = deepcopy(state_action_values_after)\n",
+ " policy = get_policy(state_action_values_after)\n",
+ "\n",
+ "print('Final Optimal Policy:')\n",
+ "# Draw the final situation\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, policy = policy, state_action_values = state_action_values, rewards = reward_structure)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "djPTKuDk76xO"
+ },
+ "source": [
+ "Finally, lets run this for a **single** episode and visualize the penguin's actions"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "pWObQf2h76xO"
+ },
+ "outputs": [],
+ "source": [
+ "def get_one_episode(n_state, state_action_values, terminal_states, gamma):\n",
+ "\n",
+ " state = np.random.randint(n_state-1)\n",
+ "\n",
+ " # Create lists to store all the states seen and actions taken throughout the single episode\n",
+ " all_states = []\n",
+ " all_actions = []\n",
+ "\n",
+ " # Initalize episode termination flag\n",
+ " done = False\n",
+ " # Initialize counter for steps in the episode\n",
+ " steps = 0\n",
+ "\n",
+ " all_states.append(state)\n",
+ "\n",
+ " while not done:\n",
+ " steps += 1\n",
+ "\n",
+ " new_state, reward, action, is_terminal = markov_decision_process_step(state, transition_probabilities_given_action, reward_structure, terminal_states)\n",
+ " all_states.append(new_state)\n",
+ " all_actions.append(action)\n",
+ "\n",
+ " state_action_values_after = q_learning_step(state_action_values, reward, state, new_state, action, is_terminal, gamma)\n",
+ "\n",
+ " # If in termination state, reset state randomly\n",
+ " if is_terminal:\n",
+ " state = np.random.randint(n_state-1)\n",
+ " print(f'Episode Terminated at {steps} Steps')\n",
+ " # Set episode termination flag\n",
+ " done = True\n",
+ " else:\n",
+ " state = new_state\n",
+ "\n",
+ " # Update the policy\n",
+ " state_action_values = deepcopy(state_action_values_after)\n",
+ " policy = get_policy(state_action_values_after)\n",
+ "\n",
+ " return all_states, all_actions, policy, state_action_values\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "P7cbCGT176xO"
+ },
+ "outputs": [],
+ "source": [
+ "def visualize_one_episode(states, actions):\n",
+ " # Define actions for visualization\n",
+ " acts = ['up', 'right', 'down', 'left']\n",
+ "\n",
+ " # Iterate over the states and actions\n",
+ " for i in range(len(states)):\n",
+ "\n",
+ " if i == 0:\n",
+ " print('Starting State:', states[i])\n",
+ "\n",
+ " elif i == len(states)-1:\n",
+ " print('Episode Done:', states[i])\n",
+ "\n",
+ " else:\n",
+ " print('State', states[i-1])\n",
+ " a = actions[i]\n",
+ " print('Action:', acts[a])\n",
+ " print('Next State:', states[i])\n",
+ "\n",
+ " # Visualize the current state using the MDP drawer\n",
+ " mdp_drawer.draw(layout, state=states[i], rewards=reward_structure, draw_state_index=True)\n",
+ " clear_output(True)\n",
+ "\n",
+ " # Pause for a short duration to allow observation\n",
+ " sleep(1.5)\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "cr98F8PT76xP"
+ },
+ "outputs": [],
+ "source": [
+ "# Initialize the state-action values to random numbers\n",
+ "np.random.seed(2)\n",
+ "n_state = transition_probabilities_given_action.shape[0]\n",
+ "n_action = transition_probabilities_given_action.shape[2]\n",
+ "state_action_values = np.random.normal(size=(n_action, n_state))\n",
+ "\n",
+ "# Hard code value of termination state of finding fish to 0\n",
+ "terminal_states = [15]\n",
+ "state_action_values[:, terminal_states] = 0\n",
+ "gamma = 0.9\n",
+ "\n",
+ "# Draw the initial setup\n",
+ "print('Initial Policy:')\n",
+ "policy = get_policy(state_action_values)\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, policy = policy, state_action_values = state_action_values, rewards = reward_structure)\n",
+ "\n",
+ "states, actions, policy, state_action_values = get_one_episode(n_state, state_action_values, terminal_states, gamma)\n",
+ "\n",
+ "print()\n",
+ "print('Final Optimal Policy:')\n",
+ "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+ "mdp_drawer.draw(layout, policy = policy, state_action_values = state_action_values, rewards = reward_structure)\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "5zBu1g3776xP"
+ },
+ "outputs": [],
+ "source": [
+ "visualize_one_episode(states, actions)"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.10.12"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
\ No newline at end of file
diff --git a/Notebooks/Chap19/19_5_Control_Variates.ipynb b/Notebooks/Chap19/19_5_Control_Variates.ipynb
new file mode 100644
index 0000000..5af4f99
--- /dev/null
+++ b/Notebooks/Chap19/19_5_Control_Variates.ipynb
@@ -0,0 +1,170 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyO6CLgMIO5bUVAMkzPT3z4y",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 19.5: Control variates**\n",
+ "\n",
+ "This notebook investigates the method of control variates as described in figure 19.16\n",
+ "\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ],
+ "metadata": {
+ "id": "OLComQyvCIJ7"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Generate from our two variables, $a$ and $b$. We are interested in estimating the mean of $a$, but we can use $b$$ to improve our estimates if it is correlated"
+ ],
+ "metadata": {
+ "id": "uwmhcAZBzTRO"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Sample from two variables with mean zero, standard deviation one, and a given correlation coefficient\n",
+ "def get_samples(n_samples, correlation_coeff=0.8):\n",
+ " a = np.random.normal(size=(1,n_samples))\n",
+ " temp = np.random.normal(size=(1, n_samples))\n",
+ " b = correlation_coeff * a + np.sqrt(1-correlation_coeff * correlation_coeff) * temp\n",
+ " return a, b"
+ ],
+ "metadata": {
+ "id": "bC8MBXPawQJ3"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "N = 10000000\n",
+ "a,b, = get_samples(N)\n",
+ "\n",
+ "# Verify that these two variables have zero mean and unit standard deviation\n",
+ "print(\"Mean of a = %3.3f, Std of a = %3.3f\"%(np.mean(a),np.std(a)))\n",
+ "print(\"Mean of b = %3.3f, Std of b = %3.3f\"%(np.mean(b),np.std(b)))"
+ ],
+ "metadata": {
+ "id": "1cT66nbRyW34"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's samples $N=10$ examples from $a$ and estimate their mean $\\mathbb{E}[a]$. We'll do this 1000000 times and then compute the variance of those estimates."
+ ],
+ "metadata": {
+ "id": "PWoYRpjS0Nlf"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "n_estimate = 1000000\n",
+ "\n",
+ "N = 5\n",
+ "\n",
+ "# TODO -- sample N examples of variable $a$\n",
+ "# Compute the mean of each\n",
+ "# Compute the mean and variance of these estimates of the mean\n",
+ "# Replace this line\n",
+ "mean_of_estimator_1 = -1; std_of_estimator_1 = -1\n",
+ "\n",
+ "print(\"Standard estimator mean = %3.3f, Standard estimator variance = %3.3f\"%(mean_of_estimator_1, std_of_estimator_1))"
+ ],
+ "metadata": {
+ "id": "n6Uem2aYzBp7"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Now let's estimate the mean $\\mathbf{E}[a]$ of $a$ by computing $a-b$ where $b$ is correlated with $a$"
+ ],
+ "metadata": {
+ "id": "F-af86z13TFc"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "n_estimate = 1000000\n",
+ "\n",
+ "N = 5\n",
+ "\n",
+ "# TODO -- sample N examples of variables $a$ and $b$\n",
+ "# Compute $c=a-b$ for each and then compute the mean of $c$\n",
+ "# Compute the mean and variance of these estimates of the mean of $c$\n",
+ "# Replace this line\n",
+ "mean_of_estimator_2 = -1; std_of_estimator_2 = -1\n",
+ "\n",
+ "print(\"Control variate estimator mean = %3.3f, Control variate estimator variance = %3.3f\"%(mean_of_estimator_2, std_of_estimator_2))"
+ ],
+ "metadata": {
+ "id": "MrEVDggY0IGU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Note that they both have a very similar mean, but the second estimator has a lower variance. \n",
+ "\n",
+ "TODO -- Experiment with different samples sizes $N$ and correlation coefficients."
+ ],
+ "metadata": {
+ "id": "Jklzkca14ofS"
+ }
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap19/Empty.png b/Notebooks/Chap19/Empty.png
new file mode 100644
index 0000000..31423bb
Binary files /dev/null and b/Notebooks/Chap19/Empty.png differ
diff --git a/Notebooks/Chap19/Fish.png b/Notebooks/Chap19/Fish.png
new file mode 100644
index 0000000..8a1a34c
Binary files /dev/null and b/Notebooks/Chap19/Fish.png differ
diff --git a/Notebooks/Chap19/Hole.png b/Notebooks/Chap19/Hole.png
new file mode 100644
index 0000000..db8cdea
Binary files /dev/null and b/Notebooks/Chap19/Hole.png differ
diff --git a/Notebooks/Chap19/Penguin.png b/Notebooks/Chap19/Penguin.png
new file mode 100644
index 0000000..1c745f6
Binary files /dev/null and b/Notebooks/Chap19/Penguin.png differ
diff --git a/Notebooks/Chap20/20_1_Random_Data.ipynb b/Notebooks/Chap20/20_1_Random_Data.ipynb
new file mode 100644
index 0000000..fa33990
--- /dev/null
+++ b/Notebooks/Chap20/20_1_Random_Data.ipynb
@@ -0,0 +1,305 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyPkSYbEjOcEmLt8tU6HxNuR",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 20.1: Random Data**\n",
+ "\n",
+ "This notebook investigates training the network with random data, as illustrated in figure 20.1.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
+ "!git clone https://github.com/greydanus/mnist1d"
+ ],
+ "metadata": {
+ "id": "D5yLObtZCi9J"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import os\n",
+ "import torch, torch.nn as nn\n",
+ "from torch.utils.data import TensorDataset, DataLoader\n",
+ "from torch.optim.lr_scheduler import StepLR\n",
+ "import matplotlib.pyplot as plt\n",
+ "import mnist1d\n",
+ "import random\n",
+ "from IPython.display import display, clear_output"
+ ],
+ "metadata": {
+ "id": "YrXWAH7sUWvU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "args = mnist1d.data.get_dataset_args()\n",
+ "data = mnist1d.data.get_dataset(args, path='./mnist1d_data.pkl', download=False, regenerate=False)\n",
+ "\n",
+ "# The training and test input and outputs are in\n",
+ "# data['x'], data['y'], data['x_test'], and data['y_test']\n",
+ "print(\"Examples in training set: {}\".format(len(data['y'])))\n",
+ "print(\"Examples in test set: {}\".format(len(data['y_test'])))\n",
+ "print(\"Length of each example: {}\".format(data['x'].shape[-1]))"
+ ],
+ "metadata": {
+ "id": "twI72ZCrCt5z"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define the network"
+ ],
+ "metadata": {
+ "id": "_sFvRDGrl4qe"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "D_i = 40 # Input dimensions\n",
+ "D_k = 300 # Hidden dimensions\n",
+ "D_o = 10 # Output dimensions\n",
+ "\n",
+ "model = nn.Sequential(\n",
+ "nn.Linear(D_i, D_k),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(D_k, D_k),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(D_k, D_k),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(D_k, D_k),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(D_k, D_o))"
+ ],
+ "metadata": {
+ "id": "FslroPJJffrh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# He initialization of weights\n",
+ "def weights_init(layer_in):\n",
+ " if isinstance(layer_in, nn.Linear):\n",
+ " nn.init.kaiming_uniform_(layer_in.weight)\n",
+ " layer_in.bias.data.fill_(0.0)"
+ ],
+ "metadata": {
+ "id": "YgLaex1pfhqz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def train_model(train_data_x, train_data_y, n_epoch):\n",
+ " # choose cross entropy loss function (equation 5.24 in the loss notes)\n",
+ " loss_function = nn.CrossEntropyLoss()\n",
+ " # construct SGD optimizer and initialize learning rate and momentum\n",
+ " optimizer = torch.optim.SGD(model.parameters(), lr = 0.02, momentum=0.9)\n",
+ " # object that decreases learning rate by half every 20 epochs\n",
+ " scheduler = StepLR(optimizer, step_size=20, gamma=0.5)\n",
+ " # create 100 dummy data points and store in data loader class\n",
+ " x_train = torch.tensor(train_data_x.transpose().astype('float32'))\n",
+ " y_train = torch.tensor(train_data_y.astype('long'))\n",
+ "\n",
+ " # load the data into a class that creates the batches\n",
+ " data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=100, shuffle=True, worker_init_fn=np.random.seed(1))\n",
+ "\n",
+ " # Initialize model weights\n",
+ " model.apply(weights_init)\n",
+ "\n",
+ " # store the loss and the % correct at each epoch\n",
+ " losses_train = np.zeros((n_epoch))\n",
+ "\n",
+ " for epoch in range(n_epoch):\n",
+ " # loop over batches\n",
+ " for i, data in enumerate(data_loader):\n",
+ " # retrieve inputs and labels for this batch\n",
+ " x_batch, y_batch = data\n",
+ " # zero the parameter gradients\n",
+ " optimizer.zero_grad()\n",
+ " # forward pass -- calculate model output\n",
+ " pred = model(x_batch)\n",
+ " # compute the loss\n",
+ " loss = loss_function(pred, y_batch)\n",
+ " # backward pass\n",
+ " loss.backward()\n",
+ " # SGD update\n",
+ " optimizer.step()\n",
+ "\n",
+ " # Run whole dataset to get statistics -- normally wouldn't do this\n",
+ " pred_train = model(x_train)\n",
+ " _, predicted_train_class = torch.max(pred_train.data, 1)\n",
+ " losses_train[epoch] = loss_function(pred_train, y_train).item()\n",
+ " if epoch % 5 == 0:\n",
+ " clear_output(wait=True)\n",
+ " display(\"Epoch %d, train loss %3.3f\"%(epoch, losses_train[epoch]))\n",
+ "\n",
+ " # tell scheduler to consider updating learning rate\n",
+ " scheduler.step()\n",
+ "\n",
+ " return losses_train"
+ ],
+ "metadata": {
+ "id": "NYw8I_3mmX5c"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Load in the data\n",
+ "train_data_x = data['x'].transpose()\n",
+ "train_data_y = data['y']\n",
+ "# Print out sizes\n",
+ "print(\"Train data: %d examples (columns), each of which has %d dimensions (rows)\"%((train_data_x.shape[1],train_data_x.shape[0])))"
+ ],
+ "metadata": {
+ "id": "4FE3HQ_vedXO"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Compute loss for proper data and plot\n",
+ "n_epoch = 60\n",
+ "loss_true_labels = train_model(train_data_x, train_data_y, n_epoch)\n",
+ "# Plot the results\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(loss_true_labels,'r-',label='true_labels')\n",
+ "# ax.set_ylim(0,0.7); ax.set_xlim(0,n_epoch)\n",
+ "ax.set_xlabel('Epoch'); ax.set_ylabel('Loss')\n",
+ "ax.legend()\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "b56wdODqemF1"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO -- Randomize the input data (train_data_x), but retain overall mean and variance\n",
+ "# Replace this line\n",
+ "train_data_x_randomized = np.copy(train_data_x)"
+ ],
+ "metadata": {
+ "id": "SbPCiiUKgTLw"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Compute loss for true labels and plot\n",
+ "n_epoch = 60\n",
+ "loss_randomized_data = train_model(train_data_x_randomized, train_data_y, n_epoch)\n",
+ "# Plot the results\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(loss_true_labels,'r-',label='true_labels')\n",
+ "ax.plot(loss_randomized_data,'b-',label='random_data')\n",
+ "# ax.set_ylim(0,0.7); ax.set_xlim(0,n_epoch)\n",
+ "ax.set_xlabel('Epoch'); ax.set_ylabel('Loss')\n",
+ "ax.legend()\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "y7CcCJvvjLnn"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO -- Permute the labels\n",
+ "# Replace this line:\n",
+ "train_data_y_permuted = np.copy(train_data_y)"
+ ],
+ "metadata": {
+ "id": "ojaMTrzKj_74"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Compute loss for true labels and plot\n",
+ "n_epoch = 60\n",
+ "loss_permuted_labels = train_model(train_data_x, train_data_y_permuted, n_epoch)\n",
+ "# Plot the results\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(loss_true_labels,'r-',label='true_labels')\n",
+ "ax.plot(loss_randomized_data,'b-',label='random_data')\n",
+ "ax.plot(loss_permuted_labels,'g-',label='random_labels')\n",
+ "# ax.set_ylim(0,0.7); ax.set_xlim(0,n_epoch)\n",
+ "ax.set_xlabel('Epoch'); ax.set_ylabel('Loss')\n",
+ "ax.legend()\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "LaYCSjyMo9LQ"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap20/20_2_Full_Batch_Gradient_Descent.ipynb b/Notebooks/Chap20/20_2_Full_Batch_Gradient_Descent.ipynb
new file mode 100644
index 0000000..3b1841a
--- /dev/null
+++ b/Notebooks/Chap20/20_2_Full_Batch_Gradient_Descent.ipynb
@@ -0,0 +1,296 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyOo4vm4MXcIvAzVlMCaLikH",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 20.2: Full Batch Gradient Descent**\n",
+ "\n",
+ "This notebook investigates training a network with full batch gradient descent as in figure 20.2. There is also a version (notebook takes a long time to run), but this didn't speed it up much for me. If you run out of CoLab time, you'll need to download the Python file and run locally.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
+ "!git clone https://github.com/greydanus/mnist1d"
+ ],
+ "metadata": {
+ "id": "D5yLObtZCi9J"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import os\n",
+ "import torch, torch.nn as nn\n",
+ "from torch.utils.data import TensorDataset, DataLoader\n",
+ "from torch.optim.lr_scheduler import StepLR\n",
+ "import matplotlib.pyplot as plt\n",
+ "import mnist1d\n",
+ "import random\n",
+ "from IPython.display import display, clear_output"
+ ],
+ "metadata": {
+ "id": "YrXWAH7sUWvU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "args = mnist1d.data.get_dataset_args()\n",
+ "data = mnist1d.data.get_dataset(args, path='./mnist1d_data.pkl', download=False, regenerate=False)\n",
+ "\n",
+ "# The training and test input and outputs are in\n",
+ "# data['x'], data['y']\n",
+ "print(\"Examples in training set: {}\".format(len(data['y'])))\n",
+ "print(\"Length of each example: {}\".format(data['x'].shape[-1]))"
+ ],
+ "metadata": {
+ "id": "twI72ZCrCt5z"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define the network"
+ ],
+ "metadata": {
+ "id": "_sFvRDGrl4qe"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Data is length forty, and there are 10 classes\n",
+ "D_i = 40\n",
+ "D_o = 10\n",
+ "\n",
+ "# create model with one hidden layer and 298 hidden units\n",
+ "model_1_layer = nn.Sequential(\n",
+ "nn.Linear(D_i, 298),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(298, D_o))\n",
+ "\n",
+ "\n",
+ "# TODO -- create model with three hidden layers and 100 hidden units per layer\n",
+ "# Replace this line\n",
+ "model_2_layer = nn.Sequential(nn.Linear(D_i, D_o))\n",
+ "\n",
+ "\n",
+ "\n",
+ "# TODO -- Create model with three hidden layers and 75 hidden units per layer\n",
+ "# Replace this line\n",
+ "model_3_layer = nn.Sequential(nn.Linear(D_i, D_o))\n",
+ "\n",
+ "\n",
+ "\n",
+ "# TODO create model with four hidden layers and 63 hidden units per layer\n",
+ "# Replace this line\n",
+ "model_4_layer = nn.Sequential(nn.Linear(D_i, D_o))\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "FslroPJJffrh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# He initialization of weights\n",
+ "def weights_init(layer_in):\n",
+ " if isinstance(layer_in, nn.Linear):\n",
+ " nn.init.kaiming_uniform_(layer_in.weight)\n",
+ " layer_in.bias.data.fill_(0.0)"
+ ],
+ "metadata": {
+ "id": "YgLaex1pfhqz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def train_model(model, train_data_x, train_data_y, n_epoch):\n",
+ " print(\"This is going to take a long time!\")\n",
+ " # choose cross entropy loss function (equation 5.24 in the loss notes)\n",
+ " loss_function = nn.CrossEntropyLoss()\n",
+ " # construct SGD optimizer and initialize learning rate to small value and momentum to 0\n",
+ " optimizer = torch.optim.SGD(model.parameters(), lr = 0.0025, momentum=0.0)\n",
+ " # create 100 dummy data points and store in data loader class\n",
+ " x_train = torch.tensor(train_data_x.transpose().astype('float32'))\n",
+ " y_train = torch.tensor(train_data_y.astype('long'))\n",
+ "\n",
+ " # load the data into a class that creates the batches -- full batch as there are 4000 examples\n",
+ " data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=4000, shuffle=False, worker_init_fn=np.random.seed(1))\n",
+ "\n",
+ " # Initialize model weights\n",
+ " model.apply(weights_init)\n",
+ "\n",
+ " # store the errors percentage at each point\n",
+ " errors_train = np.zeros((n_epoch))\n",
+ "\n",
+ " for epoch in range(n_epoch):\n",
+ " # loop over batches\n",
+ " for i, data in enumerate(data_loader):\n",
+ " # retrieve inputs and labels for this batch\n",
+ " x_batch, y_batch = data\n",
+ " # zero the parameter gradients\n",
+ " optimizer.zero_grad()\n",
+ " # forward pass -- calculate model output\n",
+ " pred = model(x_batch)\n",
+ " # compute the loss\n",
+ " loss = loss_function(pred, y_batch)\n",
+ " # Store the errors\n",
+ " _, predicted_train_class = torch.max(pred.data, 1)\n",
+ " errors_train[epoch] = 100 - 100 * (predicted_train_class == y_train).float().sum() / len(y_train)\n",
+ " # backward pass\n",
+ " loss.backward()\n",
+ " # SGD update\n",
+ " optimizer.step()\n",
+ "\n",
+ " if epoch % 10 == 0:\n",
+ " clear_output(wait=True)\n",
+ " display(\"Epoch %d, errors_train %3.3f\"%(epoch, errors_train[epoch]))\n",
+ "\n",
+ " return errors_train"
+ ],
+ "metadata": {
+ "id": "NYw8I_3mmX5c"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Load in the data\n",
+ "train_data_x = data['x'].transpose()\n",
+ "train_data_y = data['y']\n",
+ "# Print out sizes\n",
+ "print(\"Train data: %d examples (columns), each of which has %d dimensions (rows)\"%((train_data_x.shape[1],train_data_x.shape[0])))"
+ ],
+ "metadata": {
+ "id": "4FE3HQ_vedXO"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Train the models\n",
+ "errors_four_layers = train_model(model_4_layer, train_data_x, train_data_y, n_epoch=200000)"
+ ],
+ "metadata": {
+ "id": "b56wdODqemF1"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "errors_three_layers = train_model(model_3_layer, train_data_x, train_data_y, n_epoch=200000)\n"
+ ],
+ "metadata": {
+ "id": "hqY-MJVPnCBV"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "errors_two_layers = train_model(model_2_layer, train_data_x, train_data_y, n_epoch=200000)\n"
+ ],
+ "metadata": {
+ "id": "T61jfpNGnDGj"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "errors_one_layer = train_model(model_1_layer, train_data_x, train_data_y, n_epoch=500000)"
+ ],
+ "metadata": {
+ "id": "HO8ZFgYqnEQe"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Plot the results\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(errors_one_layer,'r-',label='one layer')\n",
+ "ax.plot(errors_two_layers,'g-',label='two layers')\n",
+ "ax.plot(errors_three_layers,'b-',label='three layers')\n",
+ "ax.plot(errors_four_layers,'m-',label='four layers')\n",
+ "ax.set_ylim(0,100)\n",
+ "ax.set_xlabel('Epoch'); ax.set_ylabel('Percent error')\n",
+ "ax.legend()\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "pYL0YMI5oNSR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "wJerga3M7eDw"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap20/20_2_Full_Batch_Gradient_Descent_GPU.ipynb b/Notebooks/Chap20/20_2_Full_Batch_Gradient_Descent_GPU.ipynb
new file mode 100644
index 0000000..991ce29
--- /dev/null
+++ b/Notebooks/Chap20/20_2_Full_Batch_Gradient_Descent_GPU.ipynb
@@ -0,0 +1,303 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "gpuType": "T4",
+ "authorship_tag": "ABX9TyMjPBfDONmjqTSyEQDP2gjY",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ },
+ "accelerator": "GPU"
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 20.2: Full Batch Gradient Descent**\n",
+ "\n",
+ "This notebook investigates training a network with full batch gradient descent as in figure 20.2. This is the GPU version (notebook takes a long time to run). If you are using Colab then you need to go change the runtime type to GPU on the Runtime menu. Even then, you may run out of time. If that's the case, you'll need to download the Python file and run locally.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
+ "!git clone https://github.com/greydanus/mnist1d"
+ ],
+ "metadata": {
+ "id": "D5yLObtZCi9J"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import os\n",
+ "import torch, torch.nn as nn\n",
+ "from torch.utils.data import TensorDataset, DataLoader\n",
+ "from torch.optim.lr_scheduler import StepLR\n",
+ "import matplotlib.pyplot as plt\n",
+ "import mnist1d\n",
+ "import random\n",
+ "from IPython.display import display, clear_output\n",
+ "\n",
+ "\n",
+ "# Try attaching to GPU\n",
+ "DEVICE = str(torch.device('cuda' if torch.cuda.is_available() else 'cpu'))\n",
+ "print('Using:', DEVICE)"
+ ],
+ "metadata": {
+ "id": "YrXWAH7sUWvU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "args = mnist1d.data.get_dataset_args()\n",
+ "data = mnist1d.data.get_dataset(args, path='./mnist1d_data.pkl', download=False, regenerate=False)\n",
+ "\n",
+ "# The training and test input and outputs are in\n",
+ "# data['x'], data['y']\n",
+ "print(\"Examples in training set: {}\".format(len(data['y'])))\n",
+ "print(\"Length of each example: {}\".format(data['x'].shape[-1]))"
+ ],
+ "metadata": {
+ "id": "twI72ZCrCt5z"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define the network"
+ ],
+ "metadata": {
+ "id": "_sFvRDGrl4qe"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Data is length forty, and there are 10 classes\n",
+ "D_i = 40\n",
+ "D_o = 10\n",
+ "\n",
+ "# create model with one hidden layer and 298 hidden units\n",
+ "model_1_layer = nn.Sequential(\n",
+ "nn.Linear(D_i, 298),\n",
+ "nn.ReLU(),\n",
+ "nn.Linear(298, D_o))\n",
+ "\n",
+ "\n",
+ "# TODO -- create model with three hidden layers and 100 hidden units per layer\n",
+ "# Replace this line\n",
+ "model_2_layer = nn.Sequential(nn.Linear(D_i, D_o))\n",
+ "\n",
+ "# TODO -- Create model with three hidden layers and 75 hidden units per layer\n",
+ "# Replace this line\n",
+ "model_3_layer = nn.Sequential(nn.Linear(D_i, D_o))\n",
+ "\n",
+ "# TODO create model with four hidden layers and 63 hidden units per layer\n",
+ "# Replace this line\n",
+ "model_4_layer = nn.Sequential(nn.Linear(D_i, D_o))\n"
+ ],
+ "metadata": {
+ "id": "FslroPJJffrh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# He initialization of weights\n",
+ "def weights_init(layer_in):\n",
+ " if isinstance(layer_in, nn.Linear):\n",
+ " nn.init.kaiming_uniform_(layer_in.weight)\n",
+ " layer_in.bias.data.fill_(0.0)"
+ ],
+ "metadata": {
+ "id": "YgLaex1pfhqz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def train_model(model, train_data_x, train_data_y, n_epoch, DEVICE):\n",
+ " print(\"This is going to take a long time!\")\n",
+ " # choose cross entropy loss function (equation 5.24 in the loss notes)\n",
+ " loss_function = nn.CrossEntropyLoss()\n",
+ " # construct SGD optimizer and initialize learning rate to small value and momentum to 0\n",
+ " optimizer = torch.optim.SGD(model.parameters(), lr = 0.0025, momentum=0.0)\n",
+ " # create 100 dummy data points and store in data loader class\n",
+ " x_train = torch.tensor(train_data_x.transpose(), dtype=torch.float32, device=DEVICE)\n",
+ " y_train = torch.tensor(train_data_y, dtype=torch.long, device=DEVICE)\n",
+ "\n",
+ " # load the data into a class that creates the batches -- full batch as there are 4000 examples\n",
+ " data_loader = DataLoader(TensorDataset(x_train,y_train), batch_size=4000, shuffle=False, worker_init_fn=np.random.seed(1))\n",
+ "\n",
+ " # Initialize model weights\n",
+ " model.apply(weights_init)\n",
+ "\n",
+ " # store the errors percentage at each point\n",
+ " errors_train = np.zeros((n_epoch))\n",
+ "\n",
+ " for epoch in range(n_epoch):\n",
+ " # loop over batches\n",
+ " for i, data in enumerate(data_loader):\n",
+ " # retrieve inputs and labels for this batch\n",
+ " x_batch, y_batch = data\n",
+ " # zero the parameter gradients\n",
+ " optimizer.zero_grad()\n",
+ " # forward pass -- calculate model output\n",
+ " pred = model(x_batch)\n",
+ " # compute the loss\n",
+ " loss = loss_function(pred, y_batch)\n",
+ " # Store the errors\n",
+ " _, predicted_train_class = torch.max(pred.data, 1)\n",
+ " errors_train[epoch] = 100 - 100 * (predicted_train_class == y_train).float().sum() / len(y_train)\n",
+ " # backward pass\n",
+ " loss.backward()\n",
+ " # SGD update\n",
+ " optimizer.step()\n",
+ "\n",
+ " if epoch % 10 == 0:\n",
+ " clear_output(wait=True)\n",
+ " display(\"Epoch %d, errors_train %3.3f\"%(epoch, errors_train[epoch]))\n",
+ "\n",
+ " return errors_train"
+ ],
+ "metadata": {
+ "id": "NYw8I_3mmX5c"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Load in the data\n",
+ "train_data_x = data['x'].transpose()\n",
+ "train_data_y = data['y']\n",
+ "# Print out sizes\n",
+ "print(\"Train data: %d examples (columns), each of which has %d dimensions (rows)\"%((train_data_x.shape[1],train_data_x.shape[0])))"
+ ],
+ "metadata": {
+ "id": "4FE3HQ_vedXO"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Train the four models\n",
+ "model_4_layer = model_4_layer.to(DEVICE)\n",
+ "errors_four_layers = train_model(model_4_layer, train_data_x, train_data_y, n_epoch=200000, DEVICE=DEVICE)\n"
+ ],
+ "metadata": {
+ "id": "b56wdODqemF1"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "model_3_layer = model_3_layer.to(DEVICE)\n",
+ "errors_three_layers = train_model(model_3_layer, train_data_x, train_data_y, n_epoch=200000, DEVICE=DEVICE)\n"
+ ],
+ "metadata": {
+ "id": "63WsEgDCmbB4"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "model_2_layer = model_2_layer.to(DEVICE)\n",
+ "errors_two_layers = train_model(model_2_layer, train_data_x, train_data_y, n_epoch=200000, DEVICE=DEVICE)\n"
+ ],
+ "metadata": {
+ "id": "3TfS5DaZmdCN"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "model_1_layer = model_1_layer.to(DEVICE)\n",
+ "errors_one_layer = train_model(model_1_layer, train_data_x, train_data_y, n_epoch=500000, DEVICE=DEVICE)"
+ ],
+ "metadata": {
+ "id": "3f9Z6Mh4meeA"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Plot the results\n",
+ "fig, ax = plt.subplots()\n",
+ "ax.plot(errors_one_layer,'r-',label='one layer')\n",
+ "ax.plot(errors_two_layers,'g-',label='two layers')\n",
+ "ax.plot(errors_three_layers,'b-',label='three layers')\n",
+ "ax.plot(errors_four_layers,'m-',label='four layers')\n",
+ "ax.set_ylim(0,100)\n",
+ "ax.set_xlabel('Epoch'); ax.set_ylabel('Percent error')\n",
+ "ax.legend()\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "pYL0YMI5oNSR"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [],
+ "metadata": {
+ "id": "iJem05Y03mZB"
+ },
+ "execution_count": null,
+ "outputs": []
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap20/20_3_Lottery_Tickets.ipynb b/Notebooks/Chap20/20_3_Lottery_Tickets.ipynb
new file mode 100644
index 0000000..41d5238
--- /dev/null
+++ b/Notebooks/Chap20/20_3_Lottery_Tickets.ipynb
@@ -0,0 +1,377 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "dKUcDM76bHx3"
+ },
+ "source": [
+ "# **Notebook 20.3: Lottery tickets**\n",
+ "\n",
+ "This notebook investigates the phenomenon of lottery tickets as discussed in section 20.2.7. This notebook is highly derivative of the MNIST-1D code hosted by Sam Greydanus at https://github.com/greydanus/mnist1d. \n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "id": "Sg2i1QmhKW5d"
+ },
+ "source": [
+ "# Run this if you're in a Colab\n",
+ "!git clone https://github.com/greydanus/mnist1d"
+ ],
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Lottery tickets\n",
+ "\n",
+ "Lottery tickets were first identified by [Frankle and Carbin (2018)](https://arxiv.org/abs/1803.03635). They noted that after training a network, they could set the smaller weights to zero and clamp them there and retrain to get a network that was sparser (had fewer parameters) but could actually perform better. So within the neural network there lie smaller sub-networks which are superior. If we knew what these were, we could train them from scratch, but there is currently no way of finding out."
+ ],
+ "metadata": {
+ "id": "97g8gY5XdcKR"
+ }
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "id": "KaQo7QhvXvid"
+ },
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "\n",
+ "import torch\n",
+ "import torch.nn as nn\n",
+ "import torch.nn.functional as F\n",
+ "import torch.optim as optim\n",
+ "\n",
+ "import mnist1d\n",
+ "import copy"
+ ],
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "nre26wEOfZsM"
+ },
+ "source": [
+ "## Get the MNIST1D dataset"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "id": "I-vm_gh5xTJs"
+ },
+ "source": [
+ "args = mnist1d.get_dataset_args()\n",
+ "data = mnist1d.get_dataset(args=args) # by default, this will download a pre-made dataset from the GitHub repo\n",
+ "\n",
+ "print(\"Examples in training set: {}\".format(len(data['y'])))\n",
+ "print(\"Examples in test set: {}\".format(len(data['y_test'])))\n",
+ "print(\"Length of each input: {}\".format(data['x'].shape[-1]))\n",
+ "print(\"Number of classes: {}\".format(len(data['templates']['y'])))"
+ ],
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "O2vy0FKjfDwr"
+ },
+ "source": [
+ "## Make an MLP that can be masked\n",
+ "These parameter-wise binary masks are how we will represent sparsity in this project. There's not a great PyTorch API for this yet, so here's a temporary solution."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "id": "uBx5gNW-mqH_"
+ },
+ "source": [
+ "# Class to represent linear layer where some of the weights are forced to zero.\n",
+ "class SparseLinear(torch.nn.Module):\n",
+ " def __init__(self, x_size, y_size):\n",
+ " super(SparseLinear, self).__init__()\n",
+ " self.linear = torch.nn.Linear(x_size, y_size)\n",
+ " param_vec = torch.cat([p.flatten() for p in self.parameters()])\n",
+ " self.mask = torch.ones_like(param_vec)\n",
+ "\n",
+ " def forward(self, x, apply_mask=True):\n",
+ " if apply_mask:\n",
+ " self.apply_mask()\n",
+ " return self.linear(x)\n",
+ "\n",
+ " def update_mask(self, new_mask):\n",
+ " self.mask = new_mask\n",
+ " self.apply_mask()\n",
+ "\n",
+ " def apply_mask(self):\n",
+ " self.vec2param(self.param2vec())\n",
+ "\n",
+ " def param2vec(self):\n",
+ " vec = torch.cat([p.flatten() for p in self.parameters()])\n",
+ " return self.mask * vec\n",
+ "\n",
+ " def vec2param(self, vec):\n",
+ " pointer = 0\n",
+ " for param in self.parameters():\n",
+ " param_len = np.cumprod(param.shape)[-1]\n",
+ " new_param = vec[pointer:pointer+param_len].reshape(param.shape)\n",
+ " param.data = new_param.data\n",
+ " pointer += param_len\n",
+ "\n",
+ "# A two layer residual network where the linear layers are sparse\n",
+ "class SparseMLP(torch.nn.Module):\n",
+ " def __init__(self, input_size, output_size, hidden_size=100):\n",
+ " super(SparseMLP, self).__init__()\n",
+ " self.linear1 = SparseLinear(input_size, hidden_size)\n",
+ " self.linear2 = SparseLinear(hidden_size, hidden_size)\n",
+ " self.linear3 = SparseLinear(hidden_size, output_size)\n",
+ " self.layers = [self.linear1, self.linear2, self.linear3]\n",
+ "\n",
+ " def forward(self, x):\n",
+ " h = torch.relu(self.linear1(x))\n",
+ " h = h + torch.relu(self.linear2(h))\n",
+ " h = self.linear3(h)\n",
+ " return h\n",
+ "\n",
+ " def get_layer_masks(self):\n",
+ " return [l.mask for l in self.layers]\n",
+ "\n",
+ " def set_layer_masks(self, new_masks):\n",
+ " for i, l in enumerate(self.layers):\n",
+ " l.update_mask(new_masks[i])\n",
+ "\n",
+ " def get_layer_vecs(self):\n",
+ " return [l.param2vec() for l in self.layers]\n",
+ "\n",
+ " def set_layer_vecs(self, vecs):\n",
+ " for i, l in enumerate(self.layers):\n",
+ " l.vec2param(vecs[i])"
+ ],
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "2hwmH2vIbHin"
+ },
+ "source": [
+ "Now we need a routine that takes the weights from the model and returns a mask that identifies the positions with the lowest magnitude. These will be the weights that we mask."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "id": "Md2F9WDgYSqT"
+ },
+ "source": [
+ "# absolute weights -- absolute values of all the weights from the model in a long vector\n",
+ "# percent_sparse: how much to sparsify the model\n",
+ "def get_mask(absolute_weights, percent_sparse):\n",
+ " # TODO -- Write a function that returns a mask that has a zero\n",
+ " # everywhere for the lowest \"percent_sparse\" of the absolute weights.\n",
+ " # E.g. if absolute_weights contains [5,6,0,1,7] and we want percent_sparse of 40%,\n",
+ " # we would return [1,1,0,0,1]\n",
+ " # Remember that these are torch tensors and not numpy arrays\n",
+ " # Replace this function:\n",
+ " mask = torch.ones_like(scores)\n",
+ "\n",
+ "\n",
+ " return mask"
+ ],
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "z0McGMV-a3Xo"
+ },
+ "source": [
+ "## The prune-and-retrain cycle\n",
+ "This is the core method for finding a lottery ticket. We train a model for a fixed number of epochs, prune it, and then re-train and re-prune. We repeat this cycle until we achieve the desired level of sparsity."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "id": "5idcbyA3Ylz_"
+ },
+ "source": [
+ "def find_lottery_ticket(model, dataset, args, sparsity_schedule, criteria_fn=None, **kwargs):\n",
+ "\n",
+ " criteria_fn = lambda init_params, final_params: final_params.abs()\n",
+ "\n",
+ " init_params = model.get_layer_vecs()\n",
+ " stats = {'train_losses':[], 'test_losses':[], 'train_accs':[], 'test_accs':[]}\n",
+ " models = []\n",
+ " for i, percent_sparse in enumerate(sparsity_schedule):\n",
+ "\n",
+ " # layer-wise pruning, where pruning heuristic is determined by criteria_fn\n",
+ " final_params = model.get_layer_vecs()\n",
+ " scores = [criteria_fn(ip, fp) for ip, fp in zip(init_params, final_params)]\n",
+ " masks = [get_mask(s, percent_sparse) for s in scores]\n",
+ "\n",
+ " # update model with mask and init parameters\n",
+ " model.set_layer_vecs(init_params)\n",
+ " model.set_layer_masks(masks)\n",
+ "\n",
+ " # training process\n",
+ " results = mnist1d.train_model(dataset, model, args)\n",
+ " model = results['checkpoints'][-1]\n",
+ "\n",
+ " # store stats\n",
+ " stats['train_losses'].append(results['train_losses'])\n",
+ " stats['test_losses'].append(results['test_losses'])\n",
+ " stats['train_accs'].append(results['train_acc'])\n",
+ " stats['test_accs'].append(results['test_acc'])\n",
+ "\n",
+ " # print progress\n",
+ " if (i+1) % 1 == 0:\n",
+ " print('\\tretrain #{}, sparsity {:.2f}, final_train_loss {:.3e}, max_acc {:.1f}, last_acc {:.1f}, mean_acc {:.1f}'\n",
+ " .format(i+1, percent_sparse, results['train_losses'][-1], np.max(results['test_acc']),\n",
+ " results['test_acc'][-1], np.mean(results['test_acc']) ))\n",
+ " models.append(copy.deepcopy(model))\n",
+ "\n",
+ " stats = {k: np.stack(v) for k, v in stats.items()}\n",
+ " return models, stats"
+ ],
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "m4lokvdD4DKI"
+ },
+ "source": [
+ "## Choose hyperparameters"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "id": "OUe7-b-7Yl2c"
+ },
+ "source": [
+ "# train settings\n",
+ "model_args = mnist1d.get_model_args()\n",
+ "model_args.total_steps = 1501\n",
+ "model_args.hidden_size = 500\n",
+ "model_args.print_every = 5000 # print never\n",
+ "model_args.eval_every = 100\n",
+ "model_args.learning_rate = 2e-2\n",
+ "model_args.device = str('cpu')"
+ ],
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "hVgDM5rI4J65"
+ },
+ "source": [
+ "Find the lottery ticket by repeatedly training and then pruning weights based on their magnitudes. We'll remove 1% of the weights each time. This is going to take half an hour or so. Go and have lunch or whatever."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# sparsity settings - we will train 100 models with progressively increasing sparsity\n",
+ "num_retrains = 100\n",
+ "sparsity_schedule = np.linspace(0,1.,num_retrains)\n",
+ "\n",
+ "print(\"Magnitude pruning\")\n",
+ "mnist1d.set_seed(model_args.seed)\n",
+ "model = SparseMLP(model_args.input_size, model_args.output_size, hidden_size=model_args.hidden_size)\n",
+ "\n",
+ "criteria_fn = lambda init_params, final_params: final_params.abs()\n",
+ "lott_models, lott_stats = find_lottery_ticket(model, data, model_args, sparsity_schedule, criteria_fn=criteria_fn, prune_print_every=1)"
+ ],
+ "metadata": {
+ "id": "M25YpCuS1Gn0"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "test_losses = lott_stats['test_losses'][:,-1]\n",
+ "test_accs = lott_stats['test_accs'][:,-1]\n",
+ "\n",
+ "fig,ax = plt.subplots()\n",
+ "ax.plot(sparsity_schedule, test_losses,'r-')\n",
+ "ax.plot((sparsity_schedule[0], sparsity_schedule[-1]),(test_losses[0], test_losses[0]),'k--',label='dense')\n",
+ "ax.set_xlabel('Sparsity')\n",
+ "ax.set_ylabel('Loss')\n",
+ "ax.set_xlim(0,1)\n",
+ "ax.legend()\n",
+ "plt.show()\n",
+ "\n",
+ "fig,ax = plt.subplots()\n",
+ "ax.plot(sparsity_schedule, 100-test_accs,'r-')\n",
+ "ax.plot((sparsity_schedule[0], sparsity_schedule[-1]),(100-test_accs[0], 100-test_accs[0]),'k--',label='dense')\n",
+ "ax.set_xlabel('Sparsity')\n",
+ "ax.set_ylabel('Percent Error')\n",
+ "ax.set_xlim(0,1)\n",
+ "ax.set_ylim(0,100)\n",
+ "ax.legend()\n",
+ "plt.show()\n"
+ ],
+ "metadata": {
+ "id": "TCs-kt6-3xHB"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "You should see that the test loss decreases and the errors decrease with more as the network gets sparser. The dashed line represents the original dense (unpruned) network. We have identified a simpler network that was \"inside\" the original network for which the results are superior. Of course if we make it too sparse, then it gets worse again.\n",
+ "\n",
+ "This phenomenon is explored much further in the original notebook by Sam Greydanus which can be found [here](https://github.com/greydanus/mnist1d)."
+ ],
+ "metadata": {
+ "id": "CEj5_ZEHcRpw"
+ }
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap20/20_4_Adversarial_Attacks.ipynb b/Notebooks/Chap20/20_4_Adversarial_Attacks.ipynb
new file mode 100644
index 0000000..eea7371
--- /dev/null
+++ b/Notebooks/Chap20/20_4_Adversarial_Attacks.ipynb
@@ -0,0 +1,386 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyP9amtzXsNWqkmiPUQgxzKV",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 20.4: Adversarial attacks**\n",
+ "\n",
+ "This notebook builds uses the network for classification of MNIST from Notebook 10.5. The code is adapted from https://nextjournal.com/gkoehler/pytorch-mnist, and uses the fast gradient sign attack of [Goodfellow et al. (2015)](https://arxiv.org/abs/1412.6572). Having trained, the network, we search for adversarial examples -- inputs which look very similar to class A, but are mistakenly classified as class B. We do this by starting with a correctly classified example and perturbing it according to the gradients of the network so that the output changes.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import torch\n",
+ "import torchvision\n",
+ "import torch.nn as nn\n",
+ "import torch.nn.functional as F\n",
+ "import torch.optim as optim\n",
+ "import matplotlib.pyplot as plt\n",
+ "import random"
+ ],
+ "metadata": {
+ "id": "YrXWAH7sUWvU"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run this once to load the train and test data straight into a dataloader class\n",
+ "# that will provide the batches\n",
+ "batch_size_train = 64\n",
+ "batch_size_test = 1000\n",
+ "train_loader = torch.utils.data.DataLoader(\n",
+ " torchvision.datasets.MNIST('/files/', train=True, download=True,\n",
+ " transform=torchvision.transforms.Compose([\n",
+ " torchvision.transforms.ToTensor(),\n",
+ " torchvision.transforms.Normalize(\n",
+ " (0.1307,), (0.3081,))\n",
+ " ])),\n",
+ " batch_size=batch_size_train, shuffle=True)\n",
+ "\n",
+ "test_loader = torch.utils.data.DataLoader(\n",
+ " torchvision.datasets.MNIST('/files/', train=False, download=True,\n",
+ " transform=torchvision.transforms.Compose([\n",
+ " torchvision.transforms.ToTensor(),\n",
+ " torchvision.transforms.Normalize(\n",
+ " (0.1307,), (0.3081,))\n",
+ " ])),\n",
+ " batch_size=batch_size_test, shuffle=True)"
+ ],
+ "metadata": {
+ "id": "wScBGXXFVadm"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Let's draw some of the training data\n",
+ "examples = enumerate(test_loader)\n",
+ "batch_idx, (example_data, example_targets) = next(examples)\n",
+ "\n",
+ "fig = plt.figure()\n",
+ "for i in range(6):\n",
+ " plt.subplot(2,3,i+1)\n",
+ " plt.tight_layout()\n",
+ " plt.imshow(example_data[i][0], cmap='gray', interpolation='none')\n",
+ " plt.title(\"Ground Truth: {}\".format(example_targets[i]))\n",
+ " plt.xticks([])\n",
+ " plt.yticks([])\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "8bKADvLHbiV5"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Define the network. This is a more typical way to define a network than the sequential structure. We define a class for the network, and define the parameters in the constructor. Then we use a function called forward to actually run the network. It's easy to see how you might use residual connections in this format."
+ ],
+ "metadata": {
+ "id": "_sFvRDGrl4qe"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "from os import X_OK\n",
+ "\n",
+ "class Net(nn.Module):\n",
+ " def __init__(self):\n",
+ " super(Net, self).__init__()\n",
+ " self.conv1 = nn.Conv2d(1, 10, kernel_size=5)\n",
+ " self.conv2 = nn.Conv2d(10, 20, kernel_size=5)\n",
+ " self.drop = nn.Dropout2d()\n",
+ " self.fc1 = nn.Linear(320, 50)\n",
+ " self.fc2 = nn.Linear(50, 10)\n",
+ "\n",
+ " def forward(self, x):\n",
+ " x = self.conv1(x)\n",
+ " x = F.max_pool2d(x,2)\n",
+ " x = F.relu(x)\n",
+ " x = self.conv2(x)\n",
+ " x = self.drop(x)\n",
+ " x = F.max_pool2d(x,2)\n",
+ " x = F.relu(x)\n",
+ " x = x.flatten(1)\n",
+ " x = F.relu(self.fc1(x))\n",
+ " x = self.fc2(x)\n",
+ " x = F.log_softmax(x)\n",
+ " return x"
+ ],
+ "metadata": {
+ "id": "EQkvw2KOPVl7"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# He initialization of weights\n",
+ "def weights_init(layer_in):\n",
+ " if isinstance(layer_in, nn.Linear):\n",
+ " nn.init.kaiming_uniform_(layer_in.weight)\n",
+ " layer_in.bias.data.fill_(0.0)"
+ ],
+ "metadata": {
+ "id": "qWZtkCZcU_dg"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Create network\n",
+ "model = Net()\n",
+ "# Initialize model weights\n",
+ "model.apply(weights_init)\n",
+ "# Define optimizer\n",
+ "optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.5)"
+ ],
+ "metadata": {
+ "id": "FslroPJJffrh"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Main training routine\n",
+ "def train(epoch):\n",
+ " model.train()\n",
+ " # Get each\n",
+ " for batch_idx, (data, target) in enumerate(train_loader):\n",
+ " optimizer.zero_grad()\n",
+ " output = model(data)\n",
+ " loss = F.nll_loss(output, target)\n",
+ " loss.backward()\n",
+ " optimizer.step()\n",
+ " # Store results\n",
+ " if batch_idx % 10 == 0:\n",
+ " print('Train Epoch: {} [{}/{}]\\tLoss: {:.6f}'.format(\n",
+ " epoch, batch_idx * len(data), len(train_loader.dataset), loss.item()))"
+ ],
+ "metadata": {
+ "id": "xKQd9PzkQ766"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run on test data\n",
+ "def test():\n",
+ " model.eval()\n",
+ " test_loss = 0\n",
+ " correct = 0\n",
+ " with torch.no_grad():\n",
+ " for data, target in test_loader:\n",
+ " output = model(data)\n",
+ " test_loss += F.nll_loss(output, target, size_average=False).item()\n",
+ " pred = output.data.max(1, keepdim=True)[1]\n",
+ " correct += pred.eq(target.data.view_as(pred)).sum()\n",
+ " test_loss /= len(test_loader.dataset)\n",
+ " print('\\nTest set: Avg. loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\\n'.format(\n",
+ " test_loss, correct, len(test_loader.dataset),\n",
+ " 100. * correct / len(test_loader.dataset)))"
+ ],
+ "metadata": {
+ "id": "Byn-f7qWRLxX"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Get initial performance\n",
+ "test()\n",
+ "# Train for three epochs\n",
+ "n_epochs = 3\n",
+ "for epoch in range(1, n_epochs + 1):\n",
+ " train(epoch)\n",
+ " test()"
+ ],
+ "metadata": {
+ "id": "YgLaex1pfhqz"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Run network on data we got before and show predictions\n",
+ "output = model(example_data)\n",
+ "\n",
+ "fig = plt.figure()\n",
+ "for i in range(6):\n",
+ " plt.subplot(2,3,i+1)\n",
+ " plt.tight_layout()\n",
+ " plt.imshow(example_data[i][0], cmap='gray', interpolation='none')\n",
+ " plt.title(\"Prediction: {}\".format(\n",
+ " output.data.max(1, keepdim=True)[1][i].item()))\n",
+ " plt.xticks([])\n",
+ " plt.yticks([])\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "o7fRUAy9Se1B"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "This is the code that does the adversarial attack. It is adapted from [here](https://pytorch.org/tutorials/beginner/fgsm_tutorial.html). It is an example of the fast gradient sign method (FGSM), which modifies the data by\n",
+ "\n",
+ "\n",
+ "\n",
+ "* Calculating the derivative $\\partial L/\\partial \\mathbf{x}$ of the loss $L$ with respect to the input data $\\mathbf{x}$.\n",
+ "* Finds the sign of the gradient at each point (making a tensor the same size as $\\mathbf{x}$ with a one where it was positive and minus one where it was negative. \n",
+ "* Multiplying this vector by $\\epsilon$ and adding it back to the original data\n",
+ "\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "EabuoMdP32Hd"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# FGSM attack code.\n",
+ "def fgsm_attack(x, epsilon, dLdx):\n",
+ " # TODO -- write this function\n",
+ " # Get the sign of the gradient\n",
+ " # Add epsilon times the size of gradient to x\n",
+ " # Replace this line\n",
+ " x_modified = torch.zeros_like(x)\n",
+ "\n",
+ " # Return the perturbed image\n",
+ " return x_modified"
+ ],
+ "metadata": {
+ "id": "gAX7tnld46q1"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "no_examples = 3\n",
+ "epsilon = 0.5\n",
+ "for i in range(no_examples):\n",
+ " # Reset gradients\n",
+ " optimizer.zero_grad()\n",
+ "\n",
+ " # Get the i'th data example\n",
+ " x = example_data[i,:,:,:]\n",
+ " # Add an extra dimension back to the beginning\n",
+ " x= x[None, :,:,:]\n",
+ " x.requires_grad = True\n",
+ " # Get the i'th target\n",
+ " y = torch.ones(1, dtype=torch.long) * example_targets[i]\n",
+ "\n",
+ " # Run the model\n",
+ " output = model(x)\n",
+ " # Compute the loss\n",
+ " loss = F.nll_loss(output, y)\n",
+ " # Back propagate\n",
+ " loss.backward()\n",
+ "\n",
+ " # Collect ``datagrad``\n",
+ " dLdx = x.grad.data\n",
+ "\n",
+ " # Call FGSM Attack\n",
+ " x_prime = fgsm_attack(x, epsilon, dLdx)\n",
+ "\n",
+ " # Re-classify the perturbed image\n",
+ " output_prime = model(x_prime)\n",
+ "\n",
+ " x = x.detach().numpy()\n",
+ " fig = plt.figure()\n",
+ " plt.subplot(1,2,1)\n",
+ " plt.tight_layout()\n",
+ " plt.imshow(x[0][0], cmap='gray', interpolation='none')\n",
+ " plt.title(\"Original Prediction: {}\".format(\n",
+ " output.data.max(1, keepdim=True)[1][0].item()))\n",
+ " plt.xticks([])\n",
+ " plt.yticks([])\n",
+ "\n",
+ " plt.subplot(1,2,2)\n",
+ " plt.tight_layout()\n",
+ " plt.imshow(x_prime[0][0].detach().numpy(), cmap='gray', interpolation='none')\n",
+ " plt.title(\"Perturbed Prediction: {}\".format(\n",
+ " output_prime.data.max(1, keepdim=True)[1][0].item()))\n",
+ " plt.xticks([])\n",
+ " plt.yticks([])\n",
+ "\n",
+ "plt.show()"
+ ],
+ "metadata": {
+ "id": "AuNTYWboufbm"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Although we have only added a small amount of noise, the model is fooled into thinking that these images come from a different class.\n",
+ "\n",
+ "TODO -- Modify the attack so that it iteratively perturbs the data. i.e., so we take a small step epsilon, then re-calculate the gradient and take another small step according to the new gradient signs."
+ ],
+ "metadata": {
+ "id": "vFXWK826HPQ8"
+ }
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Chap21/21_1_Bias_Mitigation.ipynb b/Notebooks/Chap21/21_1_Bias_Mitigation.ipynb
new file mode 100644
index 0000000..dba4b1a
--- /dev/null
+++ b/Notebooks/Chap21/21_1_Bias_Mitigation.ipynb
@@ -0,0 +1,457 @@
+{
+ "cells": [
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "colab_type": "text",
+ "id": "view-in-github"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ },
+ "source": [
+ "# **Notebook 21.1: Bias mitigation**\n",
+ "\n",
+ "This notebook investigates a post-processing method for bias mitigation (see figure 21.2 in the book). It based on this [blog](https://www.borealisai.com/research-blogs/tutorial1-bias-and-fairness-ai/) that I wrote for Borealis AI in 2019, which itself was derived from [this blog](https://research.google.com/bigpicture/attacking-discrimination-in-ml/) by Wattenberg, Viégas, and Hardt.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "yC_LpiJqZXEL"
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "2FYo1dWGZXgg"
+ },
+ "source": [
+ "# Worked example: loans\n",
+ "\n",
+ "Consider the example of an algorithm $c=\\text{f}[\\mathbf{x},\\boldsymbol\\phi]$ that predicts credit rating scores $c$ for loan decisions. There are two pools of loan applicants identified by the variable $p\\in\\{0,1\\}$ that we’ll describe as the blue and yellow populations. We assume that we are given historical data, so we know both the credit rating and whether the applicant actually defaulted on the loan ($y=0$) or\n",
+ " repaid it ($y=1$).\n",
+ "\n",
+ "We can now think of four groups of data corresponding to (i) the blue and yellow populations and (ii) whether they did or did not repay the loan. For each of these four groups we have a distribution of credit ratings (figure 1). In an ideal world, the two distributions for the yellow population would be exactly the same as those for the blue population. However, as figure 1 shows, this is clearly not the case here."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "O_0gGH9hZcjo"
+ },
+ "outputs": [],
+ "source": [
+ "# Class that can describe interesting curve shapes based on the input parameters\n",
+ "# Details don't matter\n",
+ "class FreqCurve:\n",
+ " def __init__(self, weight, mean1, mean2, sigma1, sigma2, prop):\n",
+ " self.mean1 = mean1\n",
+ " self.mean2 = mean2\n",
+ " self.sigma1 = sigma1\n",
+ " self.sigma2 = sigma2\n",
+ " self.prop = prop\n",
+ " self.weight = weight\n",
+ "\n",
+ " def freq(self, x):\n",
+ " return self.weight * self.prop * np.exp(-0.5 * (x-self.mean1) * (x-self.mean1) / (self.sigma1 * self.sigma1)) \\\n",
+ " * 1.0 / np.sqrt(2*np.pi*self.sigma1*self.sigma1) \\\n",
+ " + self.weight * (1-self.prop) * np.exp(-0.5 * (x-self.mean2) * (x-self.mean2) / (self.sigma2 * self.sigma2)) \\\n",
+ " * 1.0 / np.sqrt(2*np.pi*self.sigma2*self.sigma2)\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Bkp7vffBbrNW"
+ },
+ "outputs": [],
+ "source": [
+ "credit_scores = np.arange(-4,4,0.01)\n",
+ "freq_y0_p0 = FreqCurve(800, -1.5, -2.5, 0.8, 0.6, 0.6).freq(credit_scores)\n",
+ "freq_y1_p0 = FreqCurve(500, 0.1, 0.7, 1.5, 0.8, 0.4 ).freq(credit_scores)\n",
+ "freq_y0_p1 = FreqCurve(400, 0.2, -0.1, 0.8, 0.6, 0.3).freq(credit_scores)\n",
+ "freq_y1_p1 = FreqCurve(650, 0.6, 1.6, 1.2, 0.7, 0.6 ).freq(credit_scores)\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Jf7uqyRyhVdS"
+ },
+ "outputs": [],
+ "source": [
+ "\n",
+ "fig = plt.figure\n",
+ "ax = plt.subplot(2,2,1)\n",
+ "plt.tight_layout()\n",
+ "ax.plot(credit_scores, freq_y0_p0, 'b--', label='y=0 (defaulted)')\n",
+ "ax.plot(credit_scores, freq_y1_p0, 'b-', label='y=1 (repaid)')\n",
+ "ax.set_xlim(-4,4)\n",
+ "ax.set_ylim(0,500)\n",
+ "ax.set_xlabel('Credit score, $c$')\n",
+ "ax.set_ylabel('Frequency')\n",
+ "ax.set_title('Population p=0')\n",
+ "ax.legend()\n",
+ "\n",
+ "ax = plt.subplot(2,2,2)\n",
+ "plt.tight_layout()\n",
+ "ax.plot(credit_scores, freq_y0_p1, 'y--', label='y=0 (defaulted)')\n",
+ "ax.plot(credit_scores, freq_y1_p1, 'y-', label='y=1 (repaid)')\n",
+ "ax.set_xlim(-4,4)\n",
+ "ax.set_ylim(0,500)\n",
+ "ax.set_xlabel('Credit score, $c$')\n",
+ "ax.set_ylabel('Frequency')\n",
+ "ax.set_title('Population p=1')\n",
+ "ax.legend()\n",
+ "\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "CfZ-srQtmff2"
+ },
+ "source": [
+ "Why might the distributions for blue and yellow populations be different? It could be that the behaviour of the populations is identical, but the credit rating algorithm is biased; it may favor one population over another or simply be more noisy for one group. Alternatively, it could be that that the populations genuinely behave differently. In practice, the differences in blue and yellow distributions are probably attributable to a combination of these factors.\n",
+ "\n",
+ "Let’s assume that we can’t retrain the credit score prediction algorithm; our job is to adjudicate whether each individual is refused the loan ($\\hat{y}=0$)\n",
+ " or granted it ($\\hat{y}=1$). Since we only have the credit score\n",
+ " to go on, the best we can do is to assign different thresholds $\\tau_{1}$\n",
+ " and $\\tau_{2}$\n",
+ " for the blue and yellow populations so that the loan is granted if the credit score $c$ generated by the model exceeds $\\tau_0$ for the blue population and $\\tau_1$ for the yellow population."
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "569oU1OtoFz8"
+ },
+ "source": [
+ "Now let's investiate how to set these thresholds to fulfil different criteria."
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "bE7yPyuWoSUy"
+ },
+ "source": [
+ "# Blindness to protected attribute\n",
+ "\n",
+ "We'll first do the simplest possible thing. We'll choose the same threshold for both blue and yellow populations so that $\\tau_0$ = $\\tau_1$. Basically, we'll ignore what we know about the group membership. Let's see what the ramifications of that."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "WIG8I-LvoFBY"
+ },
+ "outputs": [],
+ "source": [
+ "# Set the thresholds\n",
+ "tau0 = tau1 = 0.0"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "2EvkCvVBiCBn"
+ },
+ "outputs": [],
+ "source": [
+ "def compute_probability_get_loan(credit_scores, frequencies, threshold):\n",
+ " # TODO - Write this function\n",
+ " # Return the probability that someone from this group loan based on the frequencies of each\n",
+ " # credit score for this group\n",
+ " # Replace this line:\n",
+ " prob = 0.5\n",
+ "\n",
+ "\n",
+ " return prob"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "AGT40q6_qfpv"
+ },
+ "source": [
+ "First let's see what the overall probability of getting the loan is for the yellow and blue populations."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "4nI-PR_wqWj6"
+ },
+ "outputs": [],
+ "source": [
+ "pr_get_loan_p0 = compute_probability_get_loan(credit_scores, freq_y0_p0+freq_y1_p0, tau0)\n",
+ "pr_get_loan_p1 = compute_probability_get_loan(credit_scores, freq_y0_p1+freq_y1_p1, tau1)\n",
+ "print(\"Probability blue group gets loan = %3.3f\"%(pr_get_loan_p0))\n",
+ "print(\"Probability yellow group gets loan = %3.3f\"%(pr_get_loan_p1))"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "G2pEa6h6sIyu"
+ },
+ "source": [
+ "Now let's plot a receiver operating characteristic (ROC) curve. This shows the rate of true positives $Pr(\\hat{y}=1|y=1)$ (people who got loan and paid it back) and false alarms $Pr(\\hat{y}=1|y=0)$ (people who got the loan but didn't pay it back) for all possible thresholds."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "2C7kNt3hqwiu"
+ },
+ "outputs": [],
+ "source": [
+ "def plot_roc(credit_scores, freq_y0_p0, freq_y1_p0, freq_y0_p1, freq_y1_p1, tau0, tau1):\n",
+ " true_positives_p0 = np.zeros_like(credit_scores)\n",
+ " false_alarms_p0 = np.zeros_like(credit_scores)\n",
+ " true_positives_p1 = np.zeros_like(credit_scores)\n",
+ " false_alarms_p1 = np.zeros_like(credit_scores)\n",
+ " for i in range(len(credit_scores)):\n",
+ " true_positives_p0[i] = compute_probability_get_loan(credit_scores, freq_y1_p0, credit_scores[i])\n",
+ " true_positives_p1[i] = compute_probability_get_loan(credit_scores, freq_y1_p1, credit_scores[i])\n",
+ " false_alarms_p0[i] = compute_probability_get_loan(credit_scores, freq_y0_p0, credit_scores[i])\n",
+ " false_alarms_p1[i] = compute_probability_get_loan(credit_scores, freq_y0_p1, credit_scores[i])\n",
+ "\n",
+ " true_positives_p0_tau0 = compute_probability_get_loan(credit_scores, freq_y1_p0, tau0)\n",
+ " true_positives_p1_tau1 = compute_probability_get_loan(credit_scores, freq_y1_p1, tau1)\n",
+ " false_alarms_p0_tau0 = compute_probability_get_loan(credit_scores, freq_y0_p0, tau0)\n",
+ " false_alarms_p1_tau1 = compute_probability_get_loan(credit_scores, freq_y0_p1, tau1)\n",
+ "\n",
+ " fig, ax = plt.subplots()\n",
+ " ax.plot(false_alarms_p0, true_positives_p0, 'b-')\n",
+ " ax.plot(false_alarms_p1, true_positives_p1, 'y-')\n",
+ " ax.plot(false_alarms_p0_tau0, true_positives_p0_tau0,'bo')\n",
+ " ax.plot(false_alarms_p1_tau1, true_positives_p1_tau1,'yo')\n",
+ " ax.set_xlim(0,1)\n",
+ " ax.set_ylim(0,1)\n",
+ " ax.set_xlabel('False alarms $Pr(\\hat{y}=1|y=0)$')\n",
+ " ax.set_ylabel('True positives $Pr(\\hat{y}=1|y=1)$')\n",
+ " ax.set_aspect('equal')\n",
+ "\n",
+ " plt.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "h3OOQeTsv8uS"
+ },
+ "outputs": [],
+ "source": [
+ "plot_roc(credit_scores, freq_y0_p0, freq_y1_p0, freq_y0_p1, freq_y1_p1, tau0, tau1)"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "UCObTsa57uuC"
+ },
+ "source": [
+ "On this plot, the true positive and false alarm rate for the particular thresholds ($\\tau_0=\\tau_{1}=0$) that we chose are indicated by the circles.\n",
+ "\n",
+ "This criterion is clearly not great. The blue and yellow groups get given loans at different rates overall, and (for this threshold), the false alarms and true positives are also different, so it's not even fair when we consider whether the loans really were paid back. \n",
+ "\n",
+ "TODO -- investigate setting a different threshold $\\tau_{0}=\\tau_{1}$. Is it possible to make the overall rates that loans are given the same? Is it possible to make the false alarm rates the same? Is it possible to make the true positive rates the same?"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Yhrxv5AQ-PWA"
+ },
+ "source": [
+ "# Equality of odds\n",
+ "\n",
+ "This definition of fairness proposes that the false positive and true positive rates should be the same for both populations. This also sounds reasonable, but the ROC curve shows that it is not possible for this example. There is no combination of thresholds that can achieve this because the ROC curves do not intersect. Even if they did, we would be stuck giving loans based on the particular false positive and true positive rates at the intersection which might not be desirable."
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "l6yb8vjX-gdi"
+ },
+ "source": [
+ "Demographic parity\n",
+ "\n",
+ "The thresholds can be chosen so that the same proportion of each group are classified as $\\hat{y}=1$ and given loans. We make an equal number of loans to each group despite the different tendencies of each to repay. This has the disadvantage that the true positive and false positive rates might be completely different in different populations. From the perspective of the lender, it is desirable to give loans in proportion to people’s ability to pay them back. From the perspective of an individual in a more reliable group, it may seem unfair that the other group gets offered the same number of loans despite the fact they are less reliable."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "syjZ2fn5wC9-"
+ },
+ "outputs": [],
+ "source": [
+ "# TO DO -- try to change the two thresholds so the overall probability of getting the loan is 0.6 for each group\n",
+ "# Change the values in these lines\n",
+ "tau0 = 0.3\n",
+ "tau1 = -0.1\n",
+ "\n",
+ "\n",
+ "\n",
+ "# Compute overall probability of getting loan\n",
+ "pr_get_loan_p0 = compute_probability_get_loan(credit_scores, freq_y0_p0+freq_y1_p0, tau0)\n",
+ "pr_get_loan_p1 = compute_probability_get_loan(credit_scores, freq_y0_p1+freq_y1_p1, tau1)\n",
+ "print(\"Probability blue group gets loan = %3.3f\"%(pr_get_loan_p0))\n",
+ "print(\"Probability yellow group gets loan = %3.3f\"%(pr_get_loan_p1))"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "5QrtvZZlHCJy"
+ },
+ "source": [
+ "This is good, because now both groups get roughly the same amount of loans. But hold on... let's look at the ROC curve:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "VApyl_58GUQb"
+ },
+ "outputs": [],
+ "source": [
+ "plot_roc(credit_scores, freq_y0_p0, freq_y1_p0, freq_y0_p1, freq_y1_p1, tau0, tau1)"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "_GgX_b6yIE4W"
+ },
+ "source": [
+ "The blue dot is waaay above the yellow dot. The proportion of people who are given a load and do pay it back from the blue population is much higher than that from the yellow population. From another perspective, that's unfair... it seems like the yellow population are 'allowed' to default more often than the blue. This leads to another possibility."
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "WDnaqetXHhlv"
+ },
+ "source": [
+ "# Equal opportunity:\n",
+ "\n",
+ "The thresholds are chosen so that so that the true positive rate is is the same for both population. Of the people who pay back the loan, the same proportion are offered credit in each group. In terms of the two ROC curves, it means choosing thresholds so that the vertical position on each curve is the same without regard for the horizontal position."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "zEN6HGJ7HJAZ"
+ },
+ "outputs": [],
+ "source": [
+ "# TO DO -- try to change the two thresholds so the true positive are 0.8 for each group\n",
+ "# Change the values in these lines so that both points on the curves have a height of 0.8\n",
+ "tau0 = -0.1\n",
+ "tau1 = -0.7\n",
+ "\n",
+ "\n",
+ "plot_roc(credit_scores, freq_y0_p0, freq_y1_p0, freq_y0_p1, freq_y1_p1, tau0, tau1)"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "JsyW0pBGJ24b"
+ },
+ "source": [
+ "This seems fair -- people who are given loans default at the same rate (20%) for both groups. But hold on... let's look at the overall loan rate between the two populations:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "2a5PXHeNJDjg"
+ },
+ "outputs": [],
+ "source": [
+ "# Compute overall probability of getting loan\n",
+ "pr_get_loan_p0 = compute_probability_get_loan(credit_scores, freq_y0_p0+freq_y1_p0, tau0)\n",
+ "pr_get_loan_p1 = compute_probability_get_loan(credit_scores, freq_y0_p1+freq_y1_p1, tau1)\n",
+ "print(\"Probability blue group gets loan = %3.3f\"%(pr_get_loan_p0))\n",
+ "print(\"Probability yellow group gets loan = %3.3f\"%(pr_get_loan_p1))"
+ ]
+ },
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "tZTM7N6jKC7q"
+ },
+ "source": [
+ "The conclusion from all this is that (i) definitions of fairness are quite subtle and (ii) it's not possible to satisfy them all simultaneously."
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "authorship_tag": "ABX9TyNQPfTDV6PFG7Ctcl+XVNlz",
+ "include_colab_link": true,
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3",
+ "name": "python3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/Notebooks/Chap21/21_2_Explainability.ipynb b/Notebooks/Chap21/21_2_Explainability.ipynb
new file mode 100644
index 0000000..e64cc41
--- /dev/null
+++ b/Notebooks/Chap21/21_2_Explainability.ipynb
@@ -0,0 +1,412 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "provenance": [],
+ "authorship_tag": "ABX9TyOLMPuSWpvv8BfyPV36RuJP",
+ "include_colab_link": true
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "view-in-github",
+ "colab_type": "text"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# **Notebook 21.2: Explainability**\n",
+ "\n",
+ "This notebook investigates the LIME explainability method as depicted in figure 21.3 of the book.\n",
+ "\n",
+ "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
+ "\n",
+ "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
+ ],
+ "metadata": {
+ "id": "t9vk9Elugvmi"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "import numpy.matlib\n",
+ "from matplotlib.colors import ListedColormap"
+ ],
+ "metadata": {
+ "id": "yC_LpiJqZXEL"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "First we'll build a black box model for predicting a credit score. This simulates a neural network. It takes four inputs $x1,x2,x3,x4$ in a column vector and it returns a value $y$. Let's assume that if the output $y$ is greater than 0 then you get the loan, and if the output is less than 0 then you don't get the zone."
+ ],
+ "metadata": {
+ "id": "WM6mq9KNit3j"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Details of this not important -- a hacky thing that takes four inputs and returns\n",
+ "# a scalar output\n",
+ "class BlackBoxModel:\n",
+ " def __init__(self):\n",
+ " self.n_dim = 4\n",
+ " self.n_points = 10\n",
+ " self.means = np.random.uniform(size=(self.n_dim, self.n_points))\n",
+ " self.stds = np.random.uniform(size=(self.n_dim,self.n_points))+0.1\n",
+ " self.values = np.random.normal(size=(self.n_points))/10\n",
+ " self.values = self.values - np.mean(self.values)\n",
+ "\n",
+ "\n",
+ " def intensity(self, x, mean, std, value):\n",
+ "\n",
+ " dist = (x-np.matlib.repmat(mean,1,x.shape[1])) / np.matlib.repmat(std,1,x.shape[1])\n",
+ " out = value * np.exp(-np.sum(dist*dist,axis=0))\n",
+ " out = out[None,:]\n",
+ " return out\n",
+ "\n",
+ "\n",
+ " def get_output(self,x):\n",
+ " y = np.zeros((1,x.shape[1]))\n",
+ " t_vals = np.arange(0, self.n_points-1, 0.01)\n",
+ " for t in t_vals:\n",
+ " i = np.floor(t)\n",
+ " prop = t-i\n",
+ " i = int(i)\n",
+ " y = y+ prop * self.intensity(x, self.means[:,[i]], self.stds[:,[i]], self.values[i])\n",
+ " y = y+ (1-prop) * self.intensity(x,self.means[:,[i+1]], self.stds[:,[i+1]], self.values[i+1])\n",
+ " y = np.clip(y,-10,10)\n",
+ " return y"
+ ],
+ "metadata": {
+ "id": "rt4FS42dIa9_"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Code to draw 2D slide through the four dimensional function\n",
+ "# Again, you don't really need to read this.\n",
+ "def draw_2D_slice(model, dim1, dim2, first_other_dim_value = 0.5, second_other_dim_value = 0.6):\n",
+ "\n",
+ " #Create pretty colormap as in book\n",
+ " my_colormap_vals_hex =('2a0902', '2b0a03', '2c0b04', '2d0c05', '2e0c06', '2f0d07', '300d08', '310e09', '320f0a', '330f0b', '34100b', '35110c', '36110d', '37120e', '38120f', '39130f', '3a1410', '3b1411', '3c1511', '3d1612', '3e1613', '3f1713', '401714', '411814', '421915', '431915', '451a16', '461b16', '471b17', '481c17', '491d18', '4a1d18', '4b1e19', '4c1f19', '4d1f1a', '4e201b', '50211b', '51211c', '52221c', '53231d', '54231d', '55241e', '56251e', '57261f', '58261f', '592720', '5b2821', '5c2821', '5d2922', '5e2a22', '5f2b23', '602b23', '612c24', '622d25', '632e25', '652e26', '662f26', '673027', '683027', '693128', '6a3229', '6b3329', '6c342a', '6d342a', '6f352b', '70362c', '71372c', '72372d', '73382e', '74392e', '753a2f', '763a2f', '773b30', '783c31', '7a3d31', '7b3e32', '7c3e33', '7d3f33', '7e4034', '7f4134', '804235', '814236', '824336', '834437', '854538', '864638', '874739', '88473a', '89483a', '8a493b', '8b4a3c', '8c4b3c', '8d4c3d', '8e4c3e', '8f4d3f', '904e3f', '924f40', '935041', '945141', '955242', '965343', '975343', '985444', '995545', '9a5646', '9b5746', '9c5847', '9d5948', '9e5a49', '9f5a49', 'a05b4a', 'a15c4b', 'a35d4b', 'a45e4c', 'a55f4d', 'a6604e', 'a7614e', 'a8624f', 'a96350', 'aa6451', 'ab6552', 'ac6552', 'ad6653', 'ae6754', 'af6855', 'b06955', 'b16a56', 'b26b57', 'b36c58', 'b46d59', 'b56e59', 'b66f5a', 'b7705b', 'b8715c', 'b9725d', 'ba735d', 'bb745e', 'bc755f', 'bd7660', 'be7761', 'bf7862', 'c07962', 'c17a63', 'c27b64', 'c27c65', 'c37d66', 'c47e67', 'c57f68', 'c68068', 'c78169', 'c8826a', 'c9836b', 'ca846c', 'cb856d', 'cc866e', 'cd876f', 'ce886f', 'ce8970', 'cf8a71', 'd08b72', 'd18c73', 'd28d74', 'd38e75', 'd48f76', 'd59077', 'd59178', 'd69279', 'd7937a', 'd8957b', 'd9967b', 'da977c', 'da987d', 'db997e', 'dc9a7f', 'dd9b80', 'de9c81', 'de9d82', 'df9e83', 'e09f84', 'e1a185', 'e2a286', 'e2a387', 'e3a488', 'e4a589', 'e5a68a', 'e5a78b', 'e6a88c', 'e7aa8d', 'e7ab8e', 'e8ac8f', 'e9ad90', 'eaae91', 'eaaf92', 'ebb093', 'ecb295', 'ecb396', 'edb497', 'eeb598', 'eeb699', 'efb79a', 'efb99b', 'f0ba9c', 'f1bb9d', 'f1bc9e', 'f2bd9f', 'f2bfa1', 'f3c0a2', 'f3c1a3', 'f4c2a4', 'f5c3a5', 'f5c5a6', 'f6c6a7', 'f6c7a8', 'f7c8aa', 'f7c9ab', 'f8cbac', 'f8ccad', 'f8cdae', 'f9ceb0', 'f9d0b1', 'fad1b2', 'fad2b3', 'fbd3b4', 'fbd5b6', 'fbd6b7', 'fcd7b8', 'fcd8b9', 'fcdaba', 'fddbbc', 'fddcbd', 'fddebe', 'fddfbf', 'fee0c1', 'fee1c2', 'fee3c3', 'fee4c5', 'ffe5c6', 'ffe7c7', 'ffe8c9', 'ffe9ca', 'ffebcb', 'ffeccd', 'ffedce', 'ffefcf', 'fff0d1', 'fff2d2', 'fff3d3', 'fff4d5', 'fff6d6', 'fff7d8', 'fff8d9', 'fffada', 'fffbdc', 'fffcdd', 'fffedf', 'ffffe0')\n",
+ " my_colormap_vals_dec = np.array([int(element,base=16) for element in my_colormap_vals_hex])\n",
+ " r = np.floor(my_colormap_vals_dec/(256*256))\n",
+ " g = np.floor((my_colormap_vals_dec - r *256 *256)/256)\n",
+ " b = np.floor(my_colormap_vals_dec - r * 256 *256 - g * 256)\n",
+ " my_colormap_vals = np.vstack((r,g,b)).transpose()/255.0\n",
+ " my_colormap = ListedColormap(my_colormap_vals)\n",
+ "\n",
+ " x1_vals = np.arange(0.0, 1.0, 0.01)\n",
+ " x2_vals = np.arange(0.0, 1.0, 0.01)\n",
+ " x1_mesh, x2_mesh = np.meshgrid(x1_vals,x2_vals)\n",
+ " n_vals = x1_mesh.shape[0]\n",
+ "\n",
+ " x1 = np.reshape(x1_mesh,(1,n_vals*n_vals))\n",
+ " x2 = np.reshape(x2_mesh,(1,n_vals*n_vals))\n",
+ "\n",
+ " x = np.ones((4,n_vals*n_vals))\n",
+ " x[dim1,:] = x1\n",
+ " x[dim2,:] = x2\n",
+ " if((dim1==0 and dim2 ==1) or (dim1==1 and dim2 ==0)):\n",
+ " x[2,:] = x[2,:] * first_other_dim_value\n",
+ " x[3,:] = x[3,:] * second_other_dim_value\n",
+ " message = \"$x_{2}$ = %3.3f, $x_3$=%3.3f\"%(first_other_dim_value, second_other_dim_value)\n",
+ " if((dim1==0 and dim2 ==2) or (dim1==2 and dim2 ==0)):\n",
+ " x[1,:] = x[1,:] * first_other_dim_value\n",
+ " x[3,:] = x[3,:] * second_other_dim_value\n",
+ " message = \"$x_{1}$ = %3.3f, $x_3$=%3.3f\"%(first_other_dim_value, second_other_dim_value)\n",
+ " if((dim1==0 and dim2 ==3) or (dim1==3 and dim2 ==0)):\n",
+ " x[1,:] = x[1,:] * first_other_dim_value\n",
+ " x[2,:] = x[2,:] * second_other_dim_value\n",
+ " message = \"$x_{1}$ = %3.3f, $x_2$=%3.3f\"%(first_other_dim_value, second_other_dim_value)\n",
+ " if((dim1==1 and dim2 ==2) or (dim1==2 and dim2 ==1)):\n",
+ " x[0,:] = x[0,:] * first_other_dim_value\n",
+ " x[3,:] = x[3,:] * second_other_dim_value\n",
+ " message = \"$x_{0}$ = %3.3f, $x_3$=%3.3f\"%(first_other_dim_value, second_other_dim_value)\n",
+ " if((dim1==1 and dim2 ==3) or (dim1==3 and dim2 ==1)):\n",
+ " x[0,:] = x[0,:] * first_other_dim_value\n",
+ " x[2,:] = x[2,:] * second_other_dim_value\n",
+ " message = \"$x_{0}$ = %3.3f, $x_2$=%3.3f\"%(first_other_dim_value, second_other_dim_value)\n",
+ " if((dim1==2 and dim2 ==3) or (dim1==3 and dim2 ==2)):\n",
+ " x[0,:] = x[0,:] * first_other_dim_value\n",
+ " x[1,:] = x[1,:] * second_other_dim_value\n",
+ " message = \"$x_{0}$ = %3.3f, $x_1$=%3.3f\"%(first_other_dim_value, second_other_dim_value)\n",
+ "\n",
+ " y = model.get_output(x)\n",
+ " y[0,0] = -10; y[0,1]=10 # Hack the first two values so we see whole range of colormap\n",
+ " y_mesh = np.reshape(y,(n_vals, n_vals))\n",
+ "\n",
+ "\n",
+ " fig, ax = plt.subplots()\n",
+ " fig.set_size_inches(7,7)\n",
+ " pos = ax.contourf(x1_mesh, x2_mesh, y_mesh, levels=256 ,cmap = my_colormap, vmin=-10,vmax=10.0)\n",
+ " ax.set_xlabel('Dimension x%d'%dim1);ax.set_ylabel('Dimension x%d'%dim2)\n",
+ " ax.set_title(message)\n",
+ " levels = np.array([0])\n",
+ " ax.contour(x1_mesh, x2_mesh, y_mesh, levels, cmap=my_colormap)\n",
+ " cb = fig.colorbar(pos)\n",
+ " cb.set_ticks((-10,-5,0,5,10))\n",
+ " plt.show()"
+ ],
+ "metadata": {
+ "id": "g0sosSU4RdU3"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Create an instance of our model"
+ ],
+ "metadata": {
+ "id": "RlHjBpcyjcw4"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "np.random.seed(3)\n",
+ "model = BlackBoxModel()"
+ ],
+ "metadata": {
+ "id": "-JXgQD4oT3J1"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The four inputs to the model might represent the four inputs measures of our debt, age, income etc, and the output represents the credit score.\n",
+ "\n",
+ "As a responsible model owner, we want to understand our model and make sure that it is doing something sensible. \n",
+ "\n",
+ "Unfortunately, the model describes a four dimensional function, which makes it really hard to understand (and imagine, that there could easily be hundreds of input in a real model).\n",
+ "\n",
+ "One thing that we can do it look at the effect of two of the inputs at one time. For example, we can look at how inputs 0 and 1 change when we fix dimension 2 to 0.2 and dimension 3 to 0.9. The black line represents the decision boundary (where the model predicts a credit score of zero). If we are on the wrong side of this boundary, then our loan is refused."
+ ],
+ "metadata": {
+ "id": "6LxuB6p3k-VM"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "draw_2D_slice(model,0,1,0.2,0.9)"
+ ],
+ "metadata": {
+ "id": "NblKr3W0dBJJ"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Similarly, we could look at how inputs 1 and 3 change the input when we set input 0 to 0.3 and input 2 to 0.2:"
+ ],
+ "metadata": {
+ "id": "Fp2GeFn5mIRW"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "draw_2D_slice(model,1,3,0.3,0.2)"
+ ],
+ "metadata": {
+ "id": "VzIe0py5d5Bk"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "This tells us something -- it might be good for a reality check if we had some expectations about what effect each input would have, but it's still hard to ensure that the model does something sensible everywhere, especially for models where there are thousands of inputs. Unfortunately, there are basically no good solutions to this problem at the time of writing."
+ ],
+ "metadata": {
+ "id": "A9XUV9B6m7v0"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "However, let's view this from the perspective of a customer. We can assume that the four inputs have some particular values, and see what the output is."
+ ],
+ "metadata": {
+ "id": "1w968kJQjjUm"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "x = np.array([[0.3],[0.8],[0.6],[0.3]])\n",
+ "y = model.get_output(x)\n",
+ "print(\"Your credit score is %3.3f\"%(y))\n",
+ "print(\"Sorry, your loan is refused\")"
+ ],
+ "metadata": {
+ "id": "Nr71IahkjfV3"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Well, that is bad news. Why was our loan refused? We'd like to understand what we could do to improve our credit score. One way to do this is through individual conditional expectation or ICE plots ([Goldstein et al. 2015](https://arxiv.org/abs/1309.6392)). These take shows how the model output would change as we vary a single feature. Essentially, they answer the question: what if the $k^{th}$ feature had taken another value?"
+ ],
+ "metadata": {
+ "id": "mafmi3dSkTuf"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def ice_plot(model, x, k):\n",
+ " # Get output for the input\n",
+ " y = model.get_output(x)\n",
+ "\n",
+ " # Possible values of the k'th dimension of the input\n",
+ " x_k_all = np.arange(0,1,0.001)\n",
+ " # TODO write code that varies the k'th dimension of x and runs the model on the result to create a series of outputs y\n",
+ " # Replace this line\n",
+ " y_all = np.zeros_like(x_k_all)\n",
+ "\n",
+ "\n",
+ "\n",
+ " fig, ax = plt.subplots()\n",
+ " ax.plot(x_k_all, np.squeeze(y_all), 'r-')\n",
+ " ax.plot(x[k],y,'ro') ;\n",
+ " ax.plot([0,1.0],[0.0,0.0],'k--')\n",
+ " ax.set_xlabel('Dimension x%d'%(k))\n",
+ " ax.set_ylabel('Credit score')\n",
+ " ax.set_xlim(0,1)\n",
+ " ax.set_ylim([-10,10])\n",
+ "\n",
+ " plt.show()\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "v2nNsvW-m2fb"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "ice_plot(model, x, 0)\n",
+ "ice_plot(model, x, 1)\n",
+ "ice_plot(model, x, 2)\n",
+ "ice_plot(model, x, 3)"
+ ],
+ "metadata": {
+ "id": "5I91gzfSnL9N"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "We can learn something from this. For example, decreasing the value of $x_{3}$ would be the most effective way to increase our credit score. However, this might be impossible if it is a variable we can't control like our age. Perhaps decreasing $x_0$ and $x_{1}$ might improve it further. Well... perhaps, but we don't know what is going on in our model; it might also make things much worse.\n",
+ "\n",
+ "Local interpretable model-agnostic explanations or LIME ([Ribeiro et al. 2016](https://arxiv.org/abs/1602.04938)) approximate the main machine learning model locally around a given input using a simpler model that is easier to understand. \n",
+ "\n",
+ "The principle is simple. First, we sample some points $\\mathbf{x}_{i}$ close to the input $\\mathbf{x}$ that we are interested in. Then we find the outputs $\\mathbf{y}_i$ that correspond to those inputs. Now we have a training set, and we can train any other kind of model that explains this small area of the input space. This can be a model that is much more interpretable and easier to understand such as a linear model or a tree."
+ ],
+ "metadata": {
+ "id": "RmjuAR7HojtR"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO -- Choose 100 points where each element of x is perturbed by noise sampled from a uniform distribution\n",
+ "# that takes values between [-0.05 and 0.05]. Then run these points through the model.\n",
+ "# Replace these lines\n",
+ "x_lime_train = np.matlib.repmat(x, 1, 100)\n",
+ "y_lime_train = np.ones((1,100))\n",
+ "\n",
+ "# BEGIN_ANSWER\n",
+ "x_lime_train = x_lime_train + np.random.uniform(low=-0.05,high=0.05,size=x_lime_train.shape)\n",
+ "y_lime_train = model.get_output(x_lime_train)\n",
+ "# END_ANSWER"
+ ],
+ "metadata": {
+ "id": "-GprSftsnS0M"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "We'll train a linear model:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "y = \\beta_0 + \\boldsymbol\\phi^{T}\\mathbf{x}\n",
+ "\\end{equation}"
+ ],
+ "metadata": {
+ "id": "yTFDYbqGqmcA"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# TODO -- train this model using a least squares loss\n",
+ "# to find values for the offset \\beta_0 and the four slopes in \\phi\n",
+ "# One way to do this is with sklearn.linear_model\n",
+ "# Replace this line\n",
+ "beta = 0; phi = np.zeros((1,4))\n",
+ "\n",
+ "print(phi)"
+ ],
+ "metadata": {
+ "id": "5e4VPh40qlEl"
+ },
+ "execution_count": null,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "This model is easily interpretable. The k'th coefficient tells us the how much (and in which direction) changing the value of the k'th input will change the output. This is only valid in the vicinity of the input $x$.\n",
+ "\n",
+ "Note that a more sophisticated version of LIME would weight the training points according to how close they are to the original data point of interest."
+ ],
+ "metadata": {
+ "id": "hsZHWuVWtzIK"
+ }
+ }
+ ]
+}
\ No newline at end of file
diff --git a/Notebooks/Info.txt b/Notebooks/Info.txt
new file mode 100644
index 0000000..6d71b66
--- /dev/null
+++ b/Notebooks/Info.txt
@@ -0,0 +1 @@
+This directory contains the Python notebooks referenced in the margins of the main text
diff --git a/PDFFigures/UDLAppendixPDF.zip b/PDFFigures/UDLAppendixPDF.zip
new file mode 100644
index 0000000..8b425b3
Binary files /dev/null and b/PDFFigures/UDLAppendixPDF.zip differ
diff --git a/PDFFigures/UDLChap10PDF.zip b/PDFFigures/UDLChap10PDF.zip
new file mode 100644
index 0000000..ca1fedd
Binary files /dev/null and b/PDFFigures/UDLChap10PDF.zip differ
diff --git a/PDFFigures/UDLChap11PDF.zip b/PDFFigures/UDLChap11PDF.zip
new file mode 100644
index 0000000..28528cf
Binary files /dev/null and b/PDFFigures/UDLChap11PDF.zip differ
diff --git a/PDFFigures/UDLChap12PDF.zip b/PDFFigures/UDLChap12PDF.zip
new file mode 100644
index 0000000..3430167
Binary files /dev/null and b/PDFFigures/UDLChap12PDF.zip differ
diff --git a/PDFFigures/UDLChap13PDF.zip b/PDFFigures/UDLChap13PDF.zip
new file mode 100644
index 0000000..3304e33
Binary files /dev/null and b/PDFFigures/UDLChap13PDF.zip differ
diff --git a/PDFFigures/UDLChap14PDF.zip b/PDFFigures/UDLChap14PDF.zip
new file mode 100644
index 0000000..60d9b0c
Binary files /dev/null and b/PDFFigures/UDLChap14PDF.zip differ
diff --git a/PDFFigures/UDLChap15PDF.zip b/PDFFigures/UDLChap15PDF.zip
new file mode 100644
index 0000000..05ef92e
Binary files /dev/null and b/PDFFigures/UDLChap15PDF.zip differ
diff --git a/PDFFigures/UDLChap16PDF.zip b/PDFFigures/UDLChap16PDF.zip
new file mode 100644
index 0000000..de9f785
Binary files /dev/null and b/PDFFigures/UDLChap16PDF.zip differ
diff --git a/PDFFigures/UDLChap17PDF.zip b/PDFFigures/UDLChap17PDF.zip
new file mode 100644
index 0000000..3f62a2c
Binary files /dev/null and b/PDFFigures/UDLChap17PDF.zip differ
diff --git a/PDFFigures/UDLChap18PDF.zip b/PDFFigures/UDLChap18PDF.zip
new file mode 100644
index 0000000..38f62c7
Binary files /dev/null and b/PDFFigures/UDLChap18PDF.zip differ
diff --git a/PDFFigures/UDLChap19PDF.zip b/PDFFigures/UDLChap19PDF.zip
new file mode 100644
index 0000000..f9ae5c0
Binary files /dev/null and b/PDFFigures/UDLChap19PDF.zip differ
diff --git a/PDFFigures/UDLChap1PDF.zip b/PDFFigures/UDLChap1PDF.zip
new file mode 100644
index 0000000..7340fce
Binary files /dev/null and b/PDFFigures/UDLChap1PDF.zip differ
diff --git a/PDFFigures/UDLChap20PDF.zip b/PDFFigures/UDLChap20PDF.zip
new file mode 100644
index 0000000..6be41b5
Binary files /dev/null and b/PDFFigures/UDLChap20PDF.zip differ
diff --git a/PDFFigures/UDLChap21PDF.zip b/PDFFigures/UDLChap21PDF.zip
new file mode 100644
index 0000000..27772b0
Binary files /dev/null and b/PDFFigures/UDLChap21PDF.zip differ
diff --git a/PDFFigures/UDLChap2PDF.zip b/PDFFigures/UDLChap2PDF.zip
new file mode 100644
index 0000000..7aa7681
Binary files /dev/null and b/PDFFigures/UDLChap2PDF.zip differ
diff --git a/PDFFigures/UDLChap3PDF.zip b/PDFFigures/UDLChap3PDF.zip
new file mode 100644
index 0000000..84037a8
Binary files /dev/null and b/PDFFigures/UDLChap3PDF.zip differ
diff --git a/PDFFigures/UDLChap4PDF.zip b/PDFFigures/UDLChap4PDF.zip
new file mode 100644
index 0000000..3b90803
Binary files /dev/null and b/PDFFigures/UDLChap4PDF.zip differ
diff --git a/PDFFigures/UDLChap5PDF.zip b/PDFFigures/UDLChap5PDF.zip
new file mode 100644
index 0000000..1dcf449
Binary files /dev/null and b/PDFFigures/UDLChap5PDF.zip differ
diff --git a/PDFFigures/UDLChap6PDF.zip b/PDFFigures/UDLChap6PDF.zip
new file mode 100644
index 0000000..320dfa3
Binary files /dev/null and b/PDFFigures/UDLChap6PDF.zip differ
diff --git a/PDFFigures/UDLChap7PDF.zip b/PDFFigures/UDLChap7PDF.zip
new file mode 100644
index 0000000..279db41
Binary files /dev/null and b/PDFFigures/UDLChap7PDF.zip differ
diff --git a/PDFFigures/UDLChap8PDF.zip b/PDFFigures/UDLChap8PDF.zip
new file mode 100644
index 0000000..5ad8f21
Binary files /dev/null and b/PDFFigures/UDLChap8PDF.zip differ
diff --git a/PDFFigures/UDLChap9PDF.zip b/PDFFigures/UDLChap9PDF.zip
new file mode 100644
index 0000000..5eb82f0
Binary files /dev/null and b/PDFFigures/UDLChap9PDF.zip differ
diff --git a/Slides/CM20315_01_Intro_01.pptx b/Slides/CM20315_01_Intro_01.pptx
new file mode 100644
index 0000000..3ee9d8c
Binary files /dev/null and b/Slides/CM20315_01_Intro_01.pptx differ
diff --git a/Slides/CM20315_01_Intro_02.pptx b/Slides/CM20315_01_Intro_02.pptx
new file mode 100644
index 0000000..5304ccf
Binary files /dev/null and b/Slides/CM20315_01_Intro_02.pptx differ
diff --git a/Slides/CM20315_02_Supervised.pptx b/Slides/CM20315_02_Supervised.pptx
new file mode 100644
index 0000000..ace8564
Binary files /dev/null and b/Slides/CM20315_02_Supervised.pptx differ
diff --git a/Slides/CM20315_03_Shallow.pptx b/Slides/CM20315_03_Shallow.pptx
new file mode 100644
index 0000000..ec86dde
Binary files /dev/null and b/Slides/CM20315_03_Shallow.pptx differ
diff --git a/Slides/CM20315_04_Deep.pptx b/Slides/CM20315_04_Deep.pptx
new file mode 100644
index 0000000..95c7828
Binary files /dev/null and b/Slides/CM20315_04_Deep.pptx differ
diff --git a/Slides/CM20315_05_Loss.pptx b/Slides/CM20315_05_Loss.pptx
new file mode 100644
index 0000000..447ba7a
Binary files /dev/null and b/Slides/CM20315_05_Loss.pptx differ
diff --git a/Slides/CM20315_05a_Catchup.pptx b/Slides/CM20315_05a_Catchup.pptx
new file mode 100644
index 0000000..c2cbc58
Binary files /dev/null and b/Slides/CM20315_05a_Catchup.pptx differ
diff --git a/Slides/CM20315_06_Fitting.pptx b/Slides/CM20315_06_Fitting.pptx
new file mode 100644
index 0000000..ea30387
Binary files /dev/null and b/Slides/CM20315_06_Fitting.pptx differ
diff --git a/Slides/CM20315_07_Gradients.pptx b/Slides/CM20315_07_Gradients.pptx
new file mode 100644
index 0000000..1587040
Binary files /dev/null and b/Slides/CM20315_07_Gradients.pptx differ
diff --git a/Slides/CM20315_08_Performance.pptx b/Slides/CM20315_08_Performance.pptx
new file mode 100644
index 0000000..0deeb94
Binary files /dev/null and b/Slides/CM20315_08_Performance.pptx differ
diff --git a/Slides/CM20315_09_Regularization.pptx b/Slides/CM20315_09_Regularization.pptx
new file mode 100644
index 0000000..d861b81
Binary files /dev/null and b/Slides/CM20315_09_Regularization.pptx differ
diff --git a/Slides/CM20315_10_Convolutional.pptx b/Slides/CM20315_10_Convolutional.pptx
new file mode 100644
index 0000000..24ac1fd
Binary files /dev/null and b/Slides/CM20315_10_Convolutional.pptx differ
diff --git a/Slides/CM20315_10_Convolutional_2.pptx b/Slides/CM20315_10_Convolutional_2.pptx
new file mode 100644
index 0000000..1b017c2
Binary files /dev/null and b/Slides/CM20315_10_Convolutional_2.pptx differ
diff --git a/Slides/CM20315_13_Unsupervised.pptx b/Slides/CM20315_13_Unsupervised.pptx
new file mode 100644
index 0000000..b4c3029
Binary files /dev/null and b/Slides/CM20315_13_Unsupervised.pptx differ
diff --git a/Slides/UDLChap10.pptx b/Slides/UDLChap10.pptx
new file mode 100644
index 0000000..61edf91
Binary files /dev/null and b/Slides/UDLChap10.pptx differ
diff --git a/Slides/UDLChap11.pptx b/Slides/UDLChap11.pptx
new file mode 100644
index 0000000..797ff55
Binary files /dev/null and b/Slides/UDLChap11.pptx differ
diff --git a/Slides/UDLChap12.pptx b/Slides/UDLChap12.pptx
new file mode 100644
index 0000000..dac5da0
Binary files /dev/null and b/Slides/UDLChap12.pptx differ
diff --git a/Slides/UDLChap13.pptx b/Slides/UDLChap13.pptx
new file mode 100644
index 0000000..0beab02
Binary files /dev/null and b/Slides/UDLChap13.pptx differ
diff --git a/Slides/UDLChap2.pptx b/Slides/UDLChap2.pptx
new file mode 100644
index 0000000..839d798
Binary files /dev/null and b/Slides/UDLChap2.pptx differ
diff --git a/Slides/UDLChap3.pptx b/Slides/UDLChap3.pptx
new file mode 100644
index 0000000..eb0a50b
Binary files /dev/null and b/Slides/UDLChap3.pptx differ
diff --git a/Slides/UDLChap4.pptx b/Slides/UDLChap4.pptx
new file mode 100644
index 0000000..a14f945
Binary files /dev/null and b/Slides/UDLChap4.pptx differ
diff --git a/Slides/UDLChap5.pptx b/Slides/UDLChap5.pptx
new file mode 100644
index 0000000..5817dee
Binary files /dev/null and b/Slides/UDLChap5.pptx differ
diff --git a/Slides/UDLChap6.pptx b/Slides/UDLChap6.pptx
new file mode 100644
index 0000000..15b41a9
Binary files /dev/null and b/Slides/UDLChap6.pptx differ
diff --git a/Slides/UDLChap7.pptx b/Slides/UDLChap7.pptx
new file mode 100644
index 0000000..455340a
Binary files /dev/null and b/Slides/UDLChap7.pptx differ
diff --git a/Slides/UDLChap8.pptx b/Slides/UDLChap8.pptx
new file mode 100644
index 0000000..2e1af24
Binary files /dev/null and b/Slides/UDLChap8.pptx differ
diff --git a/Slides/UDLChap9.pptx b/Slides/UDLChap9.pptx
new file mode 100644
index 0000000..6e849c4
Binary files /dev/null and b/Slides/UDLChap9.pptx differ
diff --git a/UDLCover.jpg b/UDLCover.jpg
new file mode 100644
index 0000000..414c8c7
Binary files /dev/null and b/UDLCover.jpg differ
diff --git a/UDLCoverSmall.jpg b/UDLCoverSmall.jpg
new file mode 100644
index 0000000..0916a1a
Binary files /dev/null and b/UDLCoverSmall.jpg differ
diff --git a/UDL_Answer_Booklet_Students.pdf b/UDL_Answer_Booklet_Students.pdf
new file mode 100644
index 0000000..692e6c9
Binary files /dev/null and b/UDL_Answer_Booklet_Students.pdf differ
diff --git a/UDL_Errata.pdf b/UDL_Errata.pdf
new file mode 100644
index 0000000..1c31642
Binary files /dev/null and b/UDL_Errata.pdf differ
diff --git a/index.html b/index.html
new file mode 100644
index 0000000..88499c2
--- /dev/null
+++ b/index.html
@@ -0,0 +1,406 @@
+
+
+
+
+ Download full PDF here +
2024-03-26. CC-BY-NC-ND license
+ 
+ Instructor answer booklet available with proof of credentials via MIT Press.
+Request an exam/desk copy via MIT Press.
+Figures in PDF (vector) / SVG (vector) / Powerpoint (images): +
My slides for 20 lecture undergraduate deep learning course:
+Answers to selected questions: PDF +
+Python notebooks: (Early ones more thoroughly tested than later ones!)
+ +
+ @book{prince2023understanding,
+ author = "Simon J.D. Prince",
+ title = "Understanding Deep Learning",
+ publisher = "MIT Press",
+ year = 2023,
+ url = "http://udlbook.com"
+}
+