Add files via upload

Update 3_3_Shallow_Network_Regions.ipynb

.gitignore

@@ -0,0 +1,23 @@
+# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
+
+# dependencies
+/node_modules
+/.pnp
+.pnp.js
+
+# testing
+/coverage
+
+# production
+/build
+
+# misc
+.DS_Store
+.env.local
+.env.development.local
+.env.test.local
+.env.production.local
+
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*

Blogs/BorealisGradientFlow.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyO6cFY1oR4CmbHL2QywgTXm",
+      "authorship_tag": "ABX9TyP9fLqBQPgcYJB1KXs3Scp/",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -31,7 +31,7 @@
       "source": [
         "# Gradient flow\n",
         "\n",
-        "This notebook replicates some of the results in the the Borealis AI blog on gradient flow.  \n"
+        "This notebook replicates some of the results in the the Borealis AI [blog](https://www.borealisai.com/research-blogs/gradient-flow/) on gradient flow.  \n"
       ],
       "metadata": {
         "id": "ucrRRJ4dq8_d"

Notebooks/Chap01/1_1_BackgroundMathematics.ipynb

@@ -1,18 +1,16 @@
 {
   "cells": [
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
-        "colab_type": "text",
-        "id": "view-in-github"
+        "id": "view-in-github",
+        "colab_type": "text"
       },
       "source": [
         "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/Notebooks/Chap01/1_1_BackgroundMathematics.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "s5zzKSOusPOB"
@@ -41,7 +39,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "WV2Dl6owme2d"
@@ -49,11 +46,11 @@
       "source": [
         "**Linear functions**<br> We will be using the term *linear equation* to mean a weighted sum of inputs plus an offset. If there is just one input $x$, then this is a straight line:\n",
         "\n",
-        "\\begin{equation}y=\\beta+\\omega x,\\end{equation} \n",
+        "\\begin{equation}y=\\beta+\\omega x,\\end{equation}\n",
         "\n",
         "where $\\beta$ is the y-intercept of the linear and $\\omega$ is the slope of the line. When there are two inputs $x_{1}$ and $x_{2}$, then this becomes:\n",
         "\n",
-        "\\begin{equation}y=\\beta+\\omega_1 x_1 + \\omega_2 x_2.\\end{equation} \n",
+        "\\begin{equation}y=\\beta+\\omega_1 x_1 + \\omega_2 x_2.\\end{equation}\n",
         "\n",
         "Any other functions are by definition **non-linear**.\n",
         "\n",
@@ -99,7 +96,7 @@
         "ax.plot(x,y,'r-')\n",
         "ax.set_ylim([0,10]);ax.set_xlim([0,10])\n",
         "ax.set_xlabel('x'); ax.set_ylabel('y')\n",
-        "plt.show\n",
+        "plt.show()\n",
         "\n",
         "# TODO -- experiment with changing the values of beta and omega\n",
         "# to understand what they do.  Try to make a line\n",
@@ -107,7 +104,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "AedfvD9dxShZ"
@@ -192,7 +188,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "i8tLwpls476R"
@@ -236,7 +231,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "fGzVJQ6N-mHJ"
@@ -275,11 +269,10 @@
         "# Compute with vector/matrix form\n",
         "y_vec = beta_vec+np.matmul(omega_mat, x_vec)\n",
         "print(\"Matrix/vector form\")\n",
-        "print('y1= %3.3f\\ny2 = %3.3f'%((y_vec[0],y_vec[1])))\n"
+        "print('y1= %3.3f\\ny2 = %3.3f'%((y_vec[0][0],y_vec[1][0])))\n"
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "3LGRoTMLU8ZU"
@@ -293,7 +286,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "7Y5zdKtKZAB2"
@@ -325,11 +317,10 @@
         "ax.plot(x,y,'r-')\n",
         "ax.set_ylim([0,100]);ax.set_xlim([-5,5])\n",
         "ax.set_xlabel('x'); ax.set_ylabel('exp[x]')\n",
-        "plt.show"
+        "plt.show()"
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "XyrT8257IWCu"
@@ -341,11 +332,10 @@
         "2. What is $\\exp[1]$?\n",
         "3. What is $\\exp[-\\infty]$?\n",
         "4. What is $\\exp[+\\infty]$?\n",
-        "5. A function is convex if we can draw a straight line between any two points on the function, and this line always lies above the function. Similarly, a function is concave if a straight line between any two points always lies below the function.  Is the exponential function convex or concave or neither?\n"
+        "5. A function is convex if we can draw a straight line between any two points on the function, and the line lies above the function everywhere between these two points. Similarly, a function is concave if a straight line between any two points lies below the function everywhere between these two points.  Is the exponential function convex or concave or neither?\n"
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "R6A4e5IxIWCu"
@@ -373,11 +363,10 @@
         "ax.plot(x,y,'r-')\n",
         "ax.set_ylim([-5,5]);ax.set_xlim([0,5])\n",
         "ax.set_xlabel('x'); ax.set_ylabel('$\\log[x]$')\n",
-        "plt.show"
+        "plt.show()"
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "yYWrL5AXIWCv"
@@ -397,8 +386,8 @@
   ],
   "metadata": {
     "colab": {
-      "include_colab_link": true,
-      "provenance": []
+      "provenance": [],
+      "include_colab_link": true
     },
     "kernelspec": {
       "display_name": "Python 3 (ipykernel)",

Notebooks/Chap02/2_1_Supervised_Learning.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyOmndC0N7dFV7W3Mh5ljOLl",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -197,7 +196,7 @@
       "source": [
         "# Visualizing the loss function\n",
         "\n",
-        "The above process is equivalent to to descending coordinate wise on the loss function<br>\n",
+        "The above process is equivalent to descending coordinate wise on the loss function<br>\n",
         "\n",
         "Now let's plot that function"
       ],
@@ -235,8 +234,8 @@
         "levels = 40\n",
         "ax.contour(phi0_mesh, phi1_mesh, all_losses ,levels, colors=['#80808080'])\n",
         "ax.set_ylim([1,-1])\n",
-        "ax.set_xlabel('Intercept, $\\phi_0$')\n",
-        "ax.set_ylabel('Slope, $\\phi_1$')\n",
+        "ax.set_xlabel(r'Intercept, $\\phi_0$')\n",
+        "ax.set_ylabel(r'Slope, $\\phi_1$')\n",
         "\n",
         "# Plot the position of your best fitting line on the loss function\n",
         "# It should be close to the minimum\n",

Notebooks/Chap03/3_1_Shallow_Networks_I.ipynb

@@ -1,18 +1,16 @@
 {
   "cells": [
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
-        "colab_type": "text",
-        "id": "view-in-github"
+        "id": "view-in-github",
+        "colab_type": "text"
       },
       "source": [
         "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/Notebooks/Chap03/3_1_Shallow_Networks_I.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "1Z6LB4Ybn1oN"
@@ -42,7 +40,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "wQDy9UzXpnf5"
@@ -102,8 +99,8 @@
       "source": [
         "# Define a shallow neural network with, one input, one output, and three hidden units\n",
         "def shallow_1_1_3(x, activation_fn, phi_0,phi_1,phi_2,phi_3, theta_10, theta_11, theta_20, theta_21, theta_30, theta_31):\n",
-        "  # TODO Replace the lines below to compute the three initial lines\n",
-        "  # (figure 3.3a-c) from the theta parameters.  These are the preactivations\n",
+        "  # TODO Replace the code below to compute the three initial lines\n",
+        "  # from the theta parameters (i.e. implement equations at bottom of figure 3.3a-c).  These are the preactivations\n",
         "  pre_1 = np.zeros_like(x)\n",
         "  pre_2 = np.zeros_like(x)\n",
         "  pre_3 = np.zeros_like(x)\n",
@@ -199,7 +196,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "T34bszToImKQ"
@@ -210,7 +206,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "jhaBSS8oIWSX"
@@ -269,7 +264,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "osonHsEqVp2I"
@@ -354,9 +348,8 @@
   ],
   "metadata": {
     "colab": {
-      "authorship_tag": "ABX9TyPBNztJrxnUt1ELWfm1Awa3",
-      "include_colab_link": true,
-      "provenance": []
+      "provenance": [],
+      "include_colab_link": true
     },
     "kernelspec": {
       "display_name": "Python 3",

Notebooks/Chap04/4_1_Composing_Networks.ipynb

@@ -134,7 +134,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Let's define two networks.  We'll put the prefixes n1_ and n2_ before all the variables to make it clear which network is which.  We'll just consider the inputs and outputs over the range [-1,1].  If you set the \"plot_all\" flat to True,  you can see the details of how they were created."
+        "Let's define two networks.  We'll put the prefixes n1_ and n2_ before all the variables to make it clear which network is which.  We'll just consider the inputs and outputs over the range [-1,1]."
       ],
       "metadata": {
         "id": "LxBJCObC-NTY"

Notebooks/Chap04/4_2_Clipping_functions.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyPkFrjmRAUf0fxN07RC4xMI",
+      "authorship_tag": "ABX9TyPZzptvvf7OPZai8erQ/0xT",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -127,26 +127,26 @@
         "    fig, ax = plt.subplots(3,3)\n",
         "    fig.set_size_inches(8.5, 8.5)\n",
         "    fig.tight_layout(pad=3.0)\n",
-        "    ax[0,0].plot(x,layer2_pre_1,'r-'); ax[0,0].set_ylabel('$\\psi_{10}+\\psi_{11}h_{1}+\\psi_{12}h_{2}+\\psi_{13}h_3$')\n",
-        "    ax[0,1].plot(x,layer2_pre_2,'b-'); ax[0,1].set_ylabel('$\\psi_{20}+\\psi_{21}h_{1}+\\psi_{22}h_{2}+\\psi_{23}h_3$')\n",
-        "    ax[0,2].plot(x,layer2_pre_3,'g-'); ax[0,2].set_ylabel('$\\psi_{30}+\\psi_{31}h_{1}+\\psi_{32}h_{2}+\\psi_{33}h_3$')\n",
-        "    ax[1,0].plot(x,h1_prime,'r-'); ax[1,0].set_ylabel(\"$h_{1}^{'}$\")\n",
-        "    ax[1,1].plot(x,h2_prime,'b-'); ax[1,1].set_ylabel(\"$h_{2}^{'}$\")\n",
-        "    ax[1,2].plot(x,h3_prime,'g-'); ax[1,2].set_ylabel(\"$h_{3}^{'}$\")\n",
-        "    ax[2,0].plot(x,phi1_h1_prime,'r-'); ax[2,0].set_ylabel(\"$\\phi_1 h_{1}^{'}$\")\n",
-        "    ax[2,1].plot(x,phi2_h2_prime,'b-'); ax[2,1].set_ylabel(\"$\\phi_2 h_{2}^{'}$\")\n",
-        "    ax[2,2].plot(x,phi3_h3_prime,'g-'); ax[2,2].set_ylabel(\"$\\phi_3 h_{3}^{'}$\")\n",
+        "    ax[0,0].plot(x,layer2_pre_1,'r-'); ax[0,0].set_ylabel(r'$\\psi_{10}+\\psi_{11}h_{1}+\\psi_{12}h_{2}+\\psi_{13}h_3$')\n",
+        "    ax[0,1].plot(x,layer2_pre_2,'b-'); ax[0,1].set_ylabel(r'$\\psi_{20}+\\psi_{21}h_{1}+\\psi_{22}h_{2}+\\psi_{23}h_3$')\n",
+        "    ax[0,2].plot(x,layer2_pre_3,'g-'); ax[0,2].set_ylabel(r'$\\psi_{30}+\\psi_{31}h_{1}+\\psi_{32}h_{2}+\\psi_{33}h_3$')\n",
+        "    ax[1,0].plot(x,h1_prime,'r-'); ax[1,0].set_ylabel(r\"$h_{1}^{'}$\")\n",
+        "    ax[1,1].plot(x,h2_prime,'b-'); ax[1,1].set_ylabel(r\"$h_{2}^{'}$\")\n",
+        "    ax[1,2].plot(x,h3_prime,'g-'); ax[1,2].set_ylabel(r\"$h_{3}^{'}$\")\n",
+        "    ax[2,0].plot(x,phi1_h1_prime,'r-'); ax[2,0].set_ylabel(r\"$\\phi_1 h_{1}^{'}$\")\n",
+        "    ax[2,1].plot(x,phi2_h2_prime,'b-'); ax[2,1].set_ylabel(r\"$\\phi_2 h_{2}^{'}$\")\n",
+        "    ax[2,2].plot(x,phi3_h3_prime,'g-'); ax[2,2].set_ylabel(r\"$\\phi_3 h_{3}^{'}$\")\n",
         "\n",
         "    for plot_y in range(3):\n",
         "      for plot_x in range(3):\n",
         "        ax[plot_y,plot_x].set_xlim([0,1]);ax[plot_x,plot_y].set_ylim([-1,1])\n",
         "        ax[plot_y,plot_x].set_aspect(0.5)\n",
-        "      ax[2,plot_y].set_xlabel('Input, $x$');\n",
+        "      ax[2,plot_y].set_xlabel(r'Input, $x$');\n",
         "    plt.show()\n",
         "\n",
         "    fig, ax = plt.subplots()\n",
         "    ax.plot(x,y)\n",
-        "    ax.set_xlabel('Input, $x$'); ax.set_ylabel('Output, $y$')\n",
+        "    ax.set_xlabel(r'Input, $x$'); ax.set_ylabel(r'Output, $y$')\n",
         "    ax.set_xlim([0,1]);ax.set_ylim([-1,1])\n",
         "    ax.set_aspect(0.5)\n",
         "    plt.show()"

Notebooks/Chap04/4_3_Deep_Networks.ipynb

@@ -118,7 +118,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Let's define a network.  We'll just consider the inputs and outputs over the range [-1,1].  If you set the \"plot_all\" flat to True,  you can see the details of how it was created."
+        "Let's define a network.  We'll just consider the inputs and outputs over the range [-1,1]."
       ],
       "metadata": {
         "id": "LxBJCObC-NTY"

Notebooks/Chap05/5_1_Least_Squares_Loss.ipynb

@@ -118,7 +118,7 @@
         "  ax.plot(x_model,y_model)\n",
         "  if sigma_model is not None:\n",
         "    ax.fill_between(x_model, y_model-2*sigma_model, y_model+2*sigma_model, color='lightgray')\n",
-        "  ax.set_xlabel('Input, $x$'); ax.set_ylabel('Output, $y$')\n",
+        "  ax.set_xlabel(r'Input, $x$'); ax.set_ylabel(r'Output, $y$')\n",
         "  ax.set_xlim([0,1]);ax.set_ylim([-1,1])\n",
         "  ax.set_aspect(0.5)\n",
         "  if title is not None:\n",
@@ -222,7 +222,7 @@
         "gauss_prob = normal_distribution(y_gauss, mu, sigma)\n",
         "fig, ax = plt.subplots()\n",
         "ax.plot(y_gauss, gauss_prob)\n",
-        "ax.set_xlabel('Input, $y$'); ax.set_ylabel('Probability $Pr(y)$')\n",
+        "ax.set_xlabel(r'Input, $y$'); ax.set_ylabel(r'Probability $Pr(y)$')\n",
         "ax.set_xlim([-5,5]);ax.set_ylim([0,1.0])\n",
         "plt.show()\n",
         "\n",

Notebooks/Chap05/5_2_Binary_Cross_Entropy_Loss.ipynb

@@ -119,12 +119,12 @@
         "  fig.set_size_inches(7.0, 3.5)\n",
         "  fig.tight_layout(pad=3.0)\n",
         "  ax[0].plot(x_model,out_model)\n",
-        "  ax[0].set_xlabel('Input, $x$'); ax[0].set_ylabel('Model output')\n",
+        "  ax[0].set_xlabel(r'Input, $x$'); ax[0].set_ylabel(r'Model output')\n",
         "  ax[0].set_xlim([0,1]);ax[0].set_ylim([-4,4])\n",
         "  if title is not None:\n",
         "    ax[0].set_title(title)\n",
         "  ax[1].plot(x_model,lambda_model)\n",
-        "  ax[1].set_xlabel('Input, $x$'); ax[1].set_ylabel('$\\lambda$ or Pr(y=1|x)')\n",
+        "  ax[1].set_xlabel(r'Input, $x$'); ax[1].set_ylabel(r'$\\lambda$ or Pr(y=1|x)')\n",
         "  ax[1].set_xlim([0,1]);ax[1].set_ylim([-0.05,1.05])\n",
         "  if title is not None:\n",
         "    ax[1].set_title(title)\n",

Notebooks/Chap06/6_1_Line_Search.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyN4E9Vtuk6t2BhZ0Ajv5SW3",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -67,7 +66,7 @@
         "  fig,ax = plt.subplots()\n",
         "  ax.plot(phi_plot,loss_function(phi_plot),'r-')\n",
         "  ax.set_xlim(0,1); ax.set_ylim(0,1)\n",
-        "  ax.set_xlabel('$\\phi$'); ax.set_ylabel('$L[\\phi]$')\n",
+        "  ax.set_xlabel(r'$\\phi$'); ax.set_ylabel(r'$L[\\phi]$')\n",
         "  if a is not None and b is not None and c is not None and d is not None:\n",
         "      plt.axvspan(a, d, facecolor='k', alpha=0.2)\n",
         "      ax.plot([a,a],[0,1],'b-')\n",

Notebooks/Chap06/6_2_Gradient_Descent.ipynb

@@ -265,7 +265,7 @@
         "\\frac{\\partial L}{\\partial \\phi_{1}}&\\approx & \\frac{L[\\phi_0, \\phi_1+\\delta]-L[\\phi_0, \\phi_1]}{\\delta}\n",
         "\\end{align}\n",
         "\n",
-        "We can't do this when there are many parameters;  for a million parameters, we would have to evaluate the loss function two million times, and usually computing the gradients directly is much more efficient."
+        "We can't do this when there are many parameters;  for a million parameters, we would have to evaluate the loss function one million plus one times, and usually computing the gradients directly is much more efficient."
       ]
     },
     {

Notebooks/Chap06/6_5_Adam.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyNFsCOnucz1nQt7PBEnKeTV",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -109,8 +108,8 @@
         "    ax.contour(phi0mesh, phi1mesh, loss_function, 20, colors=['#80808080'])\n",
         "    ax.plot(opt_path[0,:], opt_path[1,:],'-', color='#a0d9d3ff')\n",
         "    ax.plot(opt_path[0,:], opt_path[1,:],'.', color='#a0d9d3ff',markersize=10)\n",
-        "    ax.set_xlabel(\"$\\phi_{0}$\")\n",
-        "    ax.set_ylabel(\"$\\phi_{1}$\")\n",
+        "    ax.set_xlabel(r\"$\\phi_{0}$\")\n",
+        "    ax.set_ylabel(r\"$\\phi_{1}$\")\n",
         "    plt.show()"
       ],
       "metadata": {
@@ -169,7 +168,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Because the function changes much faster in $\\phi_1$ than in $\\phi_0$, there is no great step size to choose.  If we set the step size so that it makes sensible progress in the $\\phi_1$ direction, then it takes many iterations to converge.  If we set the step size so that we make sensible progress in the $\\phi_{0}$ direction, then the path oscillates in the $\\phi_1$ direction.  \n",
+        "Because the function changes much faster in $\\phi_1$ than in $\\phi_0$, there is no great step size to choose.  If we set the step size so that it makes sensible progress in the $\\phi_1$ direction, then it takes many iterations to converge.  If we set the step size so that we make sensible progress in the $\\phi_0$ direction, then the path oscillates in the $\\phi_1$ direction.  \n",
         "\n",
         "This motivates Adam.  At the core of Adam is the idea that we should just determine which way is downhill along each axis (i.e. left/right for $\\phi_0$ or up/down for $\\phi_1$) and move a fixed distance in that direction."
       ],
@@ -222,7 +221,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "This moves towards the minimum at a sensible speed, but we never actually converge -- the solution just bounces back and forth between the last two points.  To make it converge, we add momentum to both the estimates of the gradient and the pointwise squared gradient.  We also modify the statistics by a factor that depends on the time to make sure the progress is now slow to start with."
+        "This moves towards the minimum at a sensible speed, but we never actually converge -- the solution just bounces back and forth between the last two points.  To make it converge, we add momentum to both the estimates of the gradient and the pointwise squared gradient.  We also modify the statistics by a factor that depends on the time to make sure the progress is not slow to start with."
       ],
       "metadata": {
         "id": "_6KoKBJdGGI4"

Notebooks/Chap07/7_1_Backpropagation_in_Toy_Model.ipynb

@@ -131,7 +131,7 @@
       "source": [
         "beta0 = 1.0; beta1 = 2.0; beta2 = -3.0; beta3 = 0.4\n",
         "omega0 = 0.1; omega1 = -0.4; omega2 = 2.0; omega3 = 3.0\n",
-        "x = 2.3; y =2.0\n",
+        "x = 2.3; y = 2.0\n",
         "l_i_func = loss(x,y,beta0,beta1,beta2,beta3,omega0,omega1,omega2,omega3)\n",
         "print('l_i=%3.3f'%l_i_func)"
       ]
@@ -279,7 +279,7 @@
             "f2: true value = 7.137, your value = 0.000\n",
             "h3: true value = 0.657, your value = 0.000\n",
             "f3: true value = 2.372, your value = 0.000\n",
-            "like original = 0.139, like from forward pass = 0.000\n"
+            "l_i original = 0.139, l_i from forward pass = 0.000\n"
           ]
         }
       ],
@@ -292,7 +292,7 @@
         "print(\"f2: true value = %3.3f, your value = %3.3f\"%(7.137, f2))\n",
         "print(\"h3: true value = %3.3f, your value = %3.3f\"%(0.657, h3))\n",
         "print(\"f3: true value = %3.3f, your value = %3.3f\"%(2.372, f3))\n",
-        "print(\"like original = %3.3f, like from forward pass = %3.3f\"%(l_i_func, l_i))\n"
+        "print(\"l_i original = %3.3f, l_i from forward pass = %3.3f\"%(l_i_func, l_i))\n"
       ]
     },
     {

Notebooks/Chap07/7_2_Backpropagation.ipynb

@@ -115,9 +115,9 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Now let's run our random network.  The weight matrices $\\boldsymbol\\Omega_{1\\ldots K}$ are the entries of the list \"all_weights\" and the biases $\\boldsymbol\\beta_{1\\ldots k}$ are the entries of the list \"all_biases\"\n",
+        "Now let's run our random network.  The weight matrices $\\boldsymbol\\Omega_{1\\ldots K}$ are the entries of the list \"all_weights\" and the biases $\\boldsymbol\\beta_{1\\ldots K}$ are the entries of the list \"all_biases\"\n",
         "\n",
-        "We know that we will need the activations $\\mathbf{f}_{0\\ldots K}$ and the activations $\\mathbf{h}_{1\\ldots K}$ for the forward pass of backpropagation, so we'll store and return these as well.\n"
+        "We know that we will need the preactivations $\\mathbf{f}_{0\\ldots K}$ and the activations $\\mathbf{h}_{1\\ldots K}$ for the forward pass of backpropagation, so we'll store and return these as well.\n"
       ],
       "metadata": {
         "id": "5irtyxnLJSGX"
@@ -132,7 +132,7 @@
         "  K = len(all_weights) -1\n",
         "\n",
         "  # We'll store the pre-activations at each layer in a list \"all_f\"\n",
-        "  # and the activations in a second list[all_h].\n",
+        "  # and the activations in a second list \"all_h\".\n",
         "  all_f = [None] * (K+1)\n",
         "  all_h = [None] * (K+1)\n",
         "\n",
@@ -143,7 +143,7 @@
         "  # Run through the layers, calculating all_f[0...K-1] and all_h[1...K]\n",
         "  for layer in range(K):\n",
         "      # Update preactivations and activations at this layer according to eqn 7.16\n",
-        "      # Remmember to use np.matmul for matrrix multiplications\n",
+        "      # Remmember to use np.matmul for matrix multiplications\n",
         "      # TODO -- Replace the lines below\n",
         "      all_f[layer] = all_h[layer]\n",
         "      all_h[layer+1] = all_f[layer]\n",
@@ -166,7 +166,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Define in input\n",
+        "# Define input\n",
         "net_input = np.ones((D_i,1)) * 1.2\n",
         "# Compute network output\n",
         "net_output, all_f, all_h = compute_network_output(net_input,all_weights, all_biases)\n",
@@ -249,7 +249,7 @@
         "\n",
         "  # Now work backwards through the network\n",
         "  for layer in range(K,-1,-1):\n",
-        "    # TODO Calculate the derivatives of the loss with respect to the biases at layer this from all_dl_df[layer]. (eq 7.21)\n",
+        "    # TODO Calculate the derivatives of the loss with respect to the biases at layer from all_dl_df[layer]. (eq 7.21)\n",
         "    # NOTE!  To take a copy of matrix X, use Z=np.array(X)\n",
         "    # REPLACE THIS LINE\n",
         "    all_dl_dbiases[layer] = np.zeros_like(all_biases[layer])\n",
@@ -265,7 +265,7 @@
         "\n",
         "\n",
         "    if layer > 0:\n",
-        "      # TODO Calculate the derivatives of the loss with respect to the pre-activation f (use deriv of ReLu function, first part of last line of eq. 7.24)\n",
+        "      # TODO Calculate the derivatives of the loss with respect to the pre-activation f (use derivative of ReLu function, first part of last line of eq. 7.24)\n",
         "      # REPLACE THIS LINE\n",
         "      all_dl_df[layer-1] = np.zeros_like(all_f[layer-1])\n",
         "\n",

Notebooks/Chap07/7_3_Initialization.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyNHLXFpiSnUzAbzhtOk+bxu",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -117,10 +116,10 @@
         "def compute_network_output(net_input, all_weights, all_biases):\n",
         "\n",
         "  # Retrieve number of layers\n",
-        "  K = len(all_weights) -1\n",
+        "  K = len(all_weights)-1\n",
         "\n",
         "  # We'll store the pre-activations at each layer in a list \"all_f\"\n",
-        "  # and the activations in a second list[all_h].\n",
+        "  # and the activations in a second list \"all_h\".\n",
         "  all_f = [None] * (K+1)\n",
         "  all_h = [None] * (K+1)\n",
         "\n",
@@ -151,7 +150,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "Now let's investigate how this the size of the outputs vary as we change the initialization variance:\n"
+        "Now let's investigate how the size of the outputs vary as we change the initialization variance:\n"
       ],
       "metadata": {
         "id": "bIUrcXnOqChl"
@@ -164,7 +163,7 @@
         "K = 5\n",
         "# Number of neurons per layer\n",
         "D = 8\n",
-        "  # Input layer\n",
+        "# Input layer\n",
         "D_i = 1\n",
         "# Output layer\n",
         "D_o = 1\n",
@@ -177,7 +176,7 @@
         "data_in = np.random.normal(size=(1,n_data))\n",
         "net_output, all_f, all_h = compute_network_output(data_in, all_weights, all_biases)\n",
         "\n",
-        "for layer in range(K):\n",
+        "for layer in range(1,K+1):\n",
         "  print(\"Layer %d, std of hidden units = %3.3f\"%(layer, np.std(all_h[layer])))"
       ],
       "metadata": {
@@ -196,7 +195,7 @@
         "# Change this to 50 layers with 80 hidden units per layer\n",
         "\n",
         "# TO DO\n",
-        "# Now experiment with sigma_sq_omega to try to stop the variance of the forward computation explode"
+        "# Now experiment with sigma_sq_omega to try to stop the variance of the forward computation exploding"
       ],
       "metadata": {
         "id": "VL_SO4tar3DC"
@@ -249,6 +248,9 @@
         "\n",
         "# Main backward pass routine\n",
         "def backward_pass(all_weights, all_biases, all_f, all_h, y):\n",
+        "  # Retrieve number of layers\n",
+        "  K = len(all_weights) - 1\n",
+        "\n",
         "  # We'll store the derivatives dl_dweights and dl_dbiases in lists as well\n",
         "  all_dl_dweights = [None] * (K+1)\n",
         "  all_dl_dbiases = [None] * (K+1)\n",
@@ -297,7 +299,7 @@
         "K = 5\n",
         "# Number of neurons per layer\n",
         "D = 8\n",
-        "  # Input layer\n",
+        "# Input layer\n",
         "D_i = 1\n",
         "# Output layer\n",
         "D_o = 1\n",

Notebooks/Chap08/8_1_MNIST_1D_Performance.ipynb

@@ -46,8 +46,8 @@
     {
       "cell_type": "code",
       "source": [
-        "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
-        "!git clone https://github.com/greydanus/mnist1d"
+        "# Run this if you're in a Colab to install MNIST 1D repository\n",
+        "%pip install git+https://github.com/greydanus/mnist1d"
       ],
       "metadata": {
         "id": "ifVjS4cTOqKz"
@@ -83,6 +83,8 @@
     {
       "cell_type": "code",
       "source": [
+        "!mkdir ./sample_data\n",
+        "\n",
         "args = mnist1d.data.get_dataset_args()\n",
         "data = mnist1d.data.get_dataset(args, path='./sample_data/mnist1d_data.pkl', download=False, regenerate=False)\n",
         "\n",
@@ -136,7 +138,6 @@
         "optimizer = torch.optim.SGD(model.parameters(), lr = 0.05, momentum=0.9)\n",
         "# object that decreases learning rate by half every 10 epochs\n",
         "scheduler = StepLR(optimizer, step_size=10, gamma=0.5)\n",
-        "# create 100 dummy data points and store in data loader class\n",
         "x_train = torch.tensor(data['x'].astype('float32'))\n",
         "y_train = torch.tensor(data['y'].transpose().astype('long'))\n",
         "x_test= torch.tensor(data['x_test'].astype('float32'))\n",

Notebooks/Chap08/8_2_Bias_Variance_Trade_Off.ipynb

@@ -92,7 +92,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Draw the fitted function, together win uncertainty used to generate points\n",
+        "# Draw the fitted function, together with uncertainty used to generate points\n",
         "def plot_function(x_func, y_func, x_data=None,y_data=None, x_model = None, y_model =None, sigma_func = None, sigma_model=None):\n",
         "\n",
         "    fig,ax = plt.subplots()\n",
@@ -203,7 +203,7 @@
         "# Closed form solution\n",
         "beta, omega = fit_model_closed_form(x_data,y_data,n_hidden=3)\n",
         "\n",
-        "# Get prediction for model across graph grange\n",
+        "# Get prediction for model across graph range\n",
         "x_model = np.linspace(0,1,100);\n",
         "y_model = network(x_model, beta, omega)\n",
         "\n",
@@ -268,7 +268,7 @@
         "mean_model, std_model = get_model_mean_variance(n_data, n_datasets, n_hidden, sigma_func) ;\n",
         "\n",
         "# Plot the results\n",
-        "plot_function(x_func, y_func, x_data,y_data, x_model, mean_model, sigma_model=std_model)"
+        "plot_function(x_func, y_func, x_model=x_model, y_model=mean_model, sigma_model=std_model)"
       ],
       "metadata": {
         "id": "Wxk64t2SoX9c"
@@ -302,7 +302,7 @@
         "sigma_func = 0.3\n",
         "n_hidden = 5\n",
         "\n",
-        "# Set random seed so that get same result every time\n",
+        "# Set random seed so that we get the same result every time\n",
         "np.random.seed(1)\n",
         "\n",
         "for c_hidden in range(len(hidden_variables)):\n",

Notebooks/Chap08/8_3_Double_Descent.ipynb

@@ -5,7 +5,6 @@
     "colab": {
       "provenance": [],
       "gpuType": "T4",
-      "authorship_tag": "ABX9TyN/KUpEObCKnHZ/4Onp5sHG",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -48,8 +47,8 @@
     {
       "cell_type": "code",
       "source": [
-        "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
-        "!git clone https://github.com/greydanus/mnist1d"
+        "# Run this if you're in a Colab to install MNIST 1D repository\n",
+        "!pip install git+https://github.com/greydanus/mnist1d"
       ],
       "metadata": {
         "id": "fn9BP5N5TguP"
@@ -124,7 +123,7 @@
         "  D_k = n_hidden   # Hidden dimensions\n",
         "  D_o = 10    # Output dimensions\n",
         "\n",
-        "  # Define a model with two hidden layers of size 100\n",
+        "  # Define a model with two hidden layers\n",
         "  # And ReLU activations between them\n",
         "  model = nn.Sequential(\n",
         "  nn.Linear(D_i, D_k),\n",
@@ -157,7 +156,6 @@
         "  optimizer = torch.optim.SGD(model.parameters(), lr = 0.01, momentum=0.9)\n",
         "\n",
         "\n",
-        "  # create 100 dummy data points and store in data loader class\n",
         "  x_train = torch.tensor(data['x'].astype('float32'))\n",
         "  y_train = torch.tensor(data['y'].transpose().astype('long'))\n",
         "  x_test= torch.tensor(data['x_test'].astype('float32'))\n",

Notebooks/Chap08/8_4_High_Dimensional_Spaces.ipynb

@@ -224,7 +224,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "You should see see that by the time we get to 300 dimensions most of the volume is in the outer 1 percent. <br><br>\n",
+        "You should see that by the time we get to 300 dimensions most of the volume is in the outer 1 percent. <br><br>\n",
         "\n",
         "The conclusion of all of this is that in high dimensions you should be sceptical of your intuitions about how things work.  I have tried to visualize many things in one or two dimensions in the book, but you should also be sceptical about these visualizations!"
       ],

Notebooks/Chap09/9_1_L2_Regularization.ipynb

@@ -178,7 +178,7 @@
         "\n",
         "def draw_loss_function(compute_loss, data,  model, my_colormap, phi_iters = None):\n",
         "\n",
-        "  # Make grid of intercept/slope values to plot\n",
+        "  # Make grid of offset/frequency values to plot\n",
         "  offsets_mesh, freqs_mesh = np.meshgrid(np.arange(-10,10.0,0.1), np.arange(2.5,22.5,0.1))\n",
         "  loss_mesh = np.zeros_like(freqs_mesh)\n",
         "  # Compute loss for every set of parameters\n",
@@ -304,7 +304,7 @@
         "for c_step in range (n_steps):\n",
         "  # Do gradient descent step\n",
         "  phi_all[:,c_step+1:c_step+2] = gradient_descent_step(phi_all[:,c_step:c_step+1],data, model)\n",
-        "  # Measure loss and draw model every 4th step\n",
+        "  # Measure loss and draw model every 8th step\n",
         "  if c_step % 8 == 0:\n",
         "    loss =  compute_loss(data[0,:], data[1,:], model, phi_all[:,c_step+1:c_step+2])\n",
         "    draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
@@ -369,7 +369,7 @@
         "# Code to draw the regularization function\n",
         "def draw_reg_function():\n",
         "\n",
-        "  # Make grid of intercept/slope values to plot\n",
+        "  # Make grid of offset/frequency values to plot\n",
         "  offsets_mesh, freqs_mesh = np.meshgrid(np.arange(-10,10.0,0.1), np.arange(2.5,22.5,0.1))\n",
         "  loss_mesh = np.zeros_like(freqs_mesh)\n",
         "  # Compute loss for every set of parameters\n",
@@ -399,7 +399,7 @@
         "# Code to draw loss function with regularization\n",
         "def draw_loss_function_reg(data,  model, lambda_, my_colormap, phi_iters = None):\n",
         "\n",
-        "  # Make grid of intercept/slope values to plot\n",
+        "  # Make grid of offset/frequency values to plot\n",
         "  offsets_mesh, freqs_mesh = np.meshgrid(np.arange(-10,10.0,0.1), np.arange(2.5,22.5,0.1))\n",
         "  loss_mesh = np.zeros_like(freqs_mesh)\n",
         "  # Compute loss for every set of parameters\n",
@@ -512,7 +512,7 @@
         "for c_step in range (n_steps):\n",
         "  # Do gradient descent step\n",
         "  phi_all[:,c_step+1:c_step+2] = gradient_descent_step2(phi_all[:,c_step:c_step+1],lambda_, data, model)\n",
-        "  # Measure loss and draw model every 4th step\n",
+        "  # Measure loss and draw model every 8th step\n",
         "  if c_step % 8 == 0:\n",
         "    loss =  compute_loss2(data[0,:], data[1,:], model, phi_all[:,c_step+1:c_step+2], lambda_)\n",
         "    draw_model(data,model,phi_all[:,c_step+1], \"Iteration %d, loss = %f\"%(c_step+1,loss))\n",
@@ -528,7 +528,7 @@
     {
       "cell_type": "markdown",
       "source": [
-        "You should see that the gradient descent algorithm now finds the correct minimum.  By applying a tiny bit of domain knowledge (the parameter phi0 tends to be near zero and the parameters phi1 tends to be near 12.5), we get a better solution.  However, the cost is that this solution is slightly biased towards this prior knowledge."
+        "You should see that the gradient descent algorithm now finds the correct minimum.  By applying a tiny bit of domain knowledge (the parameter phi0 tends to be near zero and the parameter phi1 tends to be near 12.5), we get a better solution.  However, the cost is that this solution is slightly biased towards this prior knowledge."
       ],
       "metadata": {
         "id": "wrszSLrqZG4k"

Notebooks/Chap09/9_2_Implicit_Regularization.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyOR3WOJwfTlMD8eOLsPfPrz",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -140,7 +139,7 @@
         "    fig.set_size_inches(7,7)\n",
         "    ax.contourf(phi0mesh, phi1mesh, loss_function, 256, cmap=my_colormap);\n",
         "    ax.contour(phi0mesh, phi1mesh, loss_function, 20, colors=['#80808080'])\n",
-        "    ax.set_xlabel('$\\phi_{0}$'); ax.set_ylabel('$\\phi_{1}$')\n",
+        "    ax.set_xlabel(r'$\\phi_{0}$'); ax.set_ylabel(r'$\\phi_{1}$')\n",
         "\n",
         "    if grad_path_typical_lr is not None:\n",
         "        ax.plot(grad_path_typical_lr[0,:], grad_path_typical_lr[1,:],'ro-')\n",

Notebooks/Chap09/9_3_Ensembling.ipynb

@@ -52,7 +52,7 @@
         "# import libraries\n",
         "import numpy as np\n",
         "import matplotlib.pyplot as plt\n",
-        "# Define seed so get same results each time\n",
+        "# Define seed to get same results each time\n",
         "np.random.seed(1)"
       ]
     },
@@ -80,7 +80,7 @@
         "    for i in range(n_data):\n",
         "        x[i] = np.random.uniform(i/n_data, (i+1)/n_data, 1)\n",
         "\n",
-        "    # y value from running through functoin and adding noise\n",
+        "    # y value from running through function and adding noise\n",
         "    y = np.ones(n_data)\n",
         "    for i in range(n_data):\n",
         "        y[i] = true_function(x[i])\n",
@@ -96,7 +96,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Draw the fitted function, together win uncertainty used to generate points\n",
+        "# Draw the fitted function, together with uncertainty used to generate points\n",
         "def plot_function(x_func, y_func, x_data=None,y_data=None, x_model = None, y_model =None, sigma_func = None, sigma_model=None):\n",
         "\n",
         "    fig,ax = plt.subplots()\n",
@@ -137,7 +137,7 @@
         "n_data = 15\n",
         "x_data,y_data = generate_data(n_data, sigma_func)\n",
         "\n",
-        "# Plot the functinon, data and uncertainty\n",
+        "# Plot the function, data and uncertainty\n",
         "plot_function(x_func, y_func, x_data, y_data, sigma_func=sigma_func)"
       ],
       "metadata": {
@@ -216,7 +216,7 @@
         "# Closed form solution\n",
         "beta, omega = fit_model_closed_form(x_data,y_data,n_hidden=14)\n",
         "\n",
-        "# Get prediction for model across graph grange\n",
+        "# Get prediction for model across graph range\n",
         "x_model = np.linspace(0,1,100);\n",
         "y_model = network(x_model, beta, omega)\n",
         "\n",
@@ -297,7 +297,7 @@
     {
       "cell_type": "code",
       "source": [
-        "# Plot the median of the results\n",
+        "# Plot the mean of the results\n",
         "# TODO -- find the mean prediction\n",
         "# Replace this line\n",
         "y_model_mean = all_y_model[0,:]\n",

Notebooks/Chap09/9_4_Bayesian_Approach.ipynb

@@ -1,18 +1,16 @@
 {
   "cells": [
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
-        "colab_type": "text",
-        "id": "view-in-github"
+        "id": "view-in-github",
+        "colab_type": "text"
       },
       "source": [
         "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/Notebooks/Chap09/9_4_Bayesian_Approach.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "el8l05WQEO46"
@@ -38,7 +36,7 @@
         "# import libraries\n",
         "import numpy as np\n",
         "import matplotlib.pyplot as plt\n",
-        "# Define seed so get same results each time\n",
+        "# Define seed to get same results each time\n",
         "np.random.seed(1)"
       ]
     },
@@ -87,7 +85,7 @@
       },
       "outputs": [],
       "source": [
-        "# Draw the fitted function, together win uncertainty used to generate points\n",
+        "# Draw the fitted function, together with uncertainty used to generate points\n",
         "def plot_function(x_func, y_func, x_data=None,y_data=None, x_model = None, y_model =None, sigma_func = None, sigma_model=None):\n",
         "\n",
         "    fig,ax = plt.subplots()\n",
@@ -159,7 +157,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "i8T_QduzeBmM"
@@ -195,7 +192,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "JojV6ueRk49G"
@@ -211,7 +207,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "YX0O_Ciwp4W1"
@@ -225,7 +220,7 @@
         " &\\propto&\\text{Norm}_{\\boldsymbol\\phi}\\biggl[\\frac{1}{\\sigma^2}\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\mathbf{H}\\mathbf{y},\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\biggr].\n",
         "\\end{align}\n",
         "\n",
-        "In fact, since this already a normal distribution, the constant of proportionality must be one and we can write\n",
+        "In fact, since this is already a normal distribution, the constant of proportionality must be one and we can write\n",
         "\n",
         "\\begin{align}\n",
         " Pr(\\boldsymbol\\phi|\\{\\mathbf{x}_{i},\\mathbf{y}_{i}\\}) &=& \\text{Norm}_{\\boldsymbol\\phi}\\biggl[\\frac{1}{\\sigma^2}\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\mathbf{H}\\mathbf{y},\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\biggr].\n",
@@ -277,7 +272,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "GjPnlG4q0UFK"
@@ -334,7 +328,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "GiNg5EroUiUb"
@@ -343,17 +336,16 @@
         "Now we need to perform inference for a new data points $\\mathbf{x}^*$ with corresponding hidden values $\\mathbf{h}^*$.  Instead of having a single estimate of the parameters, we have a distribution over the possible parameters.  So we marginalize (integrate) over this distribution to account for all possible values:\n",
         "\n",
         "\\begin{align}\n",
-        "Pr(y^*|\\mathbf{x}^*)  &=& \\int Pr(y^{*}|\\mathbf{x}^*,\\boldsymbol\\phi)Pr(\\boldsymbol\\phi|\\{\\mathbf{x}_{i},\\mathbf{y}_{i}\\}) d\\boldsymbol\\phi\\\\\n",
-        "&=& \\int \\text{Norm}_{y^*}\\bigl[[\\mathbf{h}^{*T},1]\\boldsymbol\\phi,\\sigma^2\\bigr]\\cdot\\text{Norm}_{\\boldsymbol\\phi}\\biggl[\\frac{1}{\\sigma^2}\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\mathbf{H}\\mathbf{y},\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\biggr]d\\boldsymbol\\phi\\\\\n",
-        "&=& \\text{Norm}_{y^*}\\biggl[\\frac{1}{\\sigma^2} [\\mathbf{h}^{*T},1]\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\mathbf{H}\\mathbf{y},  [\\mathbf{h}^{*T},1]\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\n",
-        "[\\mathbf{h}^*;1]\\biggr]\n",
+        "Pr(y^*|\\mathbf{x}^*)  &= \\int Pr(y^{*}|\\mathbf{x}^*,\\boldsymbol\\phi)Pr(\\boldsymbol\\phi|\\{\\mathbf{x}_{i},\\mathbf{y}_{i}\\}) d\\boldsymbol\\phi\\\\\n",
+        "&= \\int \\text{Norm}_{y^*}\\bigl[[\\mathbf{h}^{*T},1]\\boldsymbol\\phi,\\sigma^2\\bigr]\\cdot\\text{Norm}_{\\boldsymbol\\phi}\\biggl[\\frac{1}{\\sigma^2}\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\mathbf{H}\\mathbf{y},\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\biggr]d\\boldsymbol\\phi\\\\\n",
+        "&= \\text{Norm}_{y^*}\\biggl[\\frac{1}{\\sigma^2} [\\mathbf{h}^{*T},1]\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\\mathbf{H}\\mathbf{y},  [\\mathbf{h}^{*T},1]\\left(\\frac{1}{\\sigma^2}\\mathbf{H}\\mathbf{H}^T+\\frac{1}{\\sigma_p^2}\\mathbf{I}\\right)^{-1}\n",
+        "[\\mathbf{h}^*;1]\\biggr],\n",
         "\\end{align}\n",
         "\n",
+        "where the notation $[\\mathbf{h}^{*T},1]$ is a row vector containing $\\mathbf{h}^{T}$ with a one appended to the end and $[\\mathbf{h};1 ]$ is a column vector containing $\\mathbf{h}$ with a one appended to the end.\n",
         "\n",
         "\n",
-        "\n",
-        "To compute this, we reformulated the integrand using the relations from appendices\n",
-        "C.3.3 and C.3.4 as the product of a normal distribution in $\\boldsymbol\\phi$ and a constant with respect\n",
+        "To compute this, we reformulated the integrand using the relations from appendices C.3.3 and C.3.4 as the product of a normal distribution in $\\boldsymbol\\phi$ and a constant with respect\n",
         "to $\\boldsymbol\\phi$. The integral of the normal distribution must be one, and so the final result is just the constant. This constant is itself a normal distribution in $y^*$. <br>\n",
         "\n",
         "If you feel so inclined you can work through the math of this yourself.\n",
@@ -404,7 +396,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "8Hcbe_16sK0F"
@@ -419,9 +410,8 @@
   ],
   "metadata": {
     "colab": {
-      "authorship_tag": "ABX9TyMB8B4269DVmrcLoCWrhzKF",
-      "include_colab_link": true,
-      "provenance": []
+      "provenance": [],
+      "include_colab_link": true
     },
     "kernelspec": {
       "display_name": "Python 3",

Notebooks/Chap09/9_5_Augmentation.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyM38ZVBK4/xaHk5Ys5lF6dN",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -44,8 +43,8 @@
     {
       "cell_type": "code",
       "source": [
-        "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
-        "!git clone https://github.com/greydanus/mnist1d"
+        "# Run this if you're in a Colab to install MNIST 1D repository\n",
+        "!pip install git+https://github.com/greydanus/mnist1d"
       ],
       "metadata": {
         "id": "syvgxgRr3myY"
@@ -95,7 +94,7 @@
         "D_k = 200   # Hidden dimensions\n",
         "D_o = 10    # Output dimensions\n",
         "\n",
-        "# Define a model with two hidden layers of size 100\n",
+        "# Define a model with two hidden layers of size 200\n",
         "# And ReLU activations between them\n",
         "model = nn.Sequential(\n",
         "nn.Linear(D_i, D_k),\n",
@@ -186,7 +185,7 @@
         "ax.plot(errors_test,'b-',label='test')\n",
         "ax.set_ylim(0,100); ax.set_xlim(0,n_epoch)\n",
         "ax.set_xlabel('Epoch'); ax.set_ylabel('Error')\n",
-        "ax.set_title('TrainError %3.2f, Test Error %3.2f'%(errors_train[-1],errors_test[-1]))\n",
+        "ax.set_title('Train Error %3.2f, Test Error %3.2f'%(errors_train[-1],errors_test[-1]))\n",
         "ax.legend()\n",
         "plt.show()"
       ],
@@ -233,7 +232,7 @@
       "cell_type": "code",
       "source": [
         "n_data_orig = data['x'].shape[0]\n",
-        "# We'll double the amount o fdata\n",
+        "# We'll double the amount of data\n",
         "n_data_augment = n_data_orig+4000\n",
         "augmented_x = np.zeros((n_data_augment, D_i))\n",
         "augmented_y = np.zeros(n_data_augment)\n",

Notebooks/Chap10/10_2_Convolution_for_MNIST_1D.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyNJodaaCLMRWL9vTl8B/iLI",
+      "authorship_tag": "ABX9TyNb46PJB/CC1pcHGfjpUUZg",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -45,8 +45,8 @@
     {
       "cell_type": "code",
       "source": [
-        "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
-        "!git clone https://github.com/greydanus/mnist1d"
+        "# Run this if you're in a Colab to install MNIST 1D repository\n",
+        "!pip install git+https://github.com/greydanus/mnist1d"
       ],
       "metadata": {
         "id": "D5yLObtZCi9J"

Notebooks/Chap11/11_1_Shattered_Gradients.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyMrF4rB2hTKq7XzLuYsURdL",
+      "authorship_tag": "ABX9TyP3VmRg51U+7NCfSYjRRrgv",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -235,7 +235,7 @@
         "# Finite difference calculation\n",
         "dydx_fd = (y2-y1)/delta\n",
         "\n",
-        "print(\"Gradient calculation=%f, Finite difference gradient=%f\"%(dydx,dydx_fd))\n"
+        "print(\"Gradient calculation=%f, Finite difference gradient=%f\"%(dydx.squeeze(),dydx_fd.squeeze()))\n"
       ],
       "metadata": {
         "id": "KJpQPVd36Haq"
@@ -267,8 +267,8 @@
         "  fig,ax = plt.subplots()\n",
         "  ax.plot(np.squeeze(x_in), np.squeeze(dydx), 'b-')\n",
         "  ax.set_xlim(-2,2)\n",
-        "  ax.set_xlabel('Input, $x$')\n",
-        "  ax.set_ylabel('Gradient, $dy/dx$')\n",
+        "  ax.set_xlabel(r'Input, $x$')\n",
+        "  ax.set_ylabel(r'Gradient, $dy/dx$')\n",
         "  ax.set_title('No layers = %d'%(K))\n",
         "  plt.show()"
       ],

Notebooks/Chap11/11_2_Residual_Networks.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyMXS3SPB4cS/4qxix0lH/Hq",
+      "authorship_tag": "ABX9TyNIY8tswL9e48d5D53aSmHO",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -45,8 +45,8 @@
     {
       "cell_type": "code",
       "source": [
-        "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
-        "!git clone https://github.com/greydanus/mnist1d"
+        "# Run this if you're in a Colab to install MNIST 1D repository\n",
+        "!pip install git+https://github.com/greydanus/mnist1d"
       ],
       "metadata": {
         "id": "D5yLObtZCi9J"

Notebooks/Chap11/11_3_Batch_Normalization.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyPVeAd3eDpEOCFh8CVyr1zz",
+      "authorship_tag": "ABX9TyPx2mM2zTHmDJeKeiE1RymT",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -45,8 +45,8 @@
     {
       "cell_type": "code",
       "source": [
-        "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
-        "!git clone https://github.com/greydanus/mnist1d"
+        "# Run this if you're in a Colab to install MNIST 1D repository\n",
+        "!pip install git+https://github.com/greydanus/mnist1d"
       ],
       "metadata": {
         "id": "D5yLObtZCi9J"

Notebooks/Chap12/12_2_Multihead_Self_Attention.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyMSk8qTqDYqFnRJVZKlsue0",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -147,9 +146,7 @@
         "  exp_values = np.exp(data_in) ;\n",
         "  # Sum over columns\n",
         "  denom = np.sum(exp_values, axis = 0);\n",
-        "  # Replicate denominator to N rows\n",
-        "  denom = np.matmul(np.ones((data_in.shape[0],1)), denom[np.newaxis,:])\n",
-        "  # Compute softmax\n",
+        "  # Compute softmax (numpy broadcasts denominator to all rows automatically)\n",
         "  softmax = exp_values / denom\n",
         "  # return the answer\n",
         "  return softmax"

Notebooks/Chap13/13_2_Graph_Classification.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyOMSGUFWT+YN0fwYHpMmHJM",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -99,7 +98,7 @@
         "\n",
         "# TODO -- Define node matrix\n",
         "# There will be 9 nodes and 118 possible chemical elements\n",
-        "# so we'll define a 9x118 matrix.  Each column represents one\n",
+        "# so we'll define a 118x9 matrix.  Each column represents one\n",
         "# node and is a one-hot vector (i.e. all zeros, except a single one at the\n",
         "# chemical number of the element).\n",
         "# Chemical numbers:  Hydrogen-->1, Carbon-->6,  Oxygen-->8\n",

Notebooks/Chap15/15_2_Wasserstein_Distance.ipynb

@@ -4,7 +4,6 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyNyLnpoXgKN+RGCuTUszCAZ",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -129,7 +128,7 @@
     {
       "cell_type": "code",
       "source": [
-        "draw_2D_heatmap(dist_mat,'Distance $|i-j|$', my_colormap)"
+        "draw_2D_heatmap(dist_mat,r'Distance $|i-j|$', my_colormap)"
       ],
       "metadata": {
         "id": "G0HFPBXyHT6V"
@@ -153,9 +152,9 @@
       "cell_type": "code",
       "source": [
         "# TODO:  Now construct the matrix A that has the initial distribution constraints\n",
-        "# so that Ap=b where p is the transport plan P vectorized rows first so p = np.flatten(P)\n",
+        "# so that A @ TPFlat=b where TPFlat is the transport plan TP vectorized rows first so TPFlat = np.flatten(TP)\n",
         "# Replace this line:\n",
-        "A = np.zeros((20,100))\n"
+        "A = np.zeros((20,100))"
       ],
       "metadata": {
         "id": "7KrybL96IuNW"
@@ -197,8 +196,8 @@
     {
       "cell_type": "code",
       "source": [
-        "P = np.array(opt.x).reshape(10,10)\n",
-        "draw_2D_heatmap(P,'Transport plan $\\mathbf{P}$', my_colormap)"
+        "TP = np.array(opt.x).reshape(10,10)\n",
+        "draw_2D_heatmap(TP,r'Transport plan $\\mathbf{P}$', my_colormap)"
       ],
       "metadata": {
         "id": "nZGfkrbRV_D0"
@@ -218,8 +217,9 @@
     {
       "cell_type": "code",
       "source": [
-        "was = np.sum(P * dist_mat)\n",
-        "print(\"Wasserstein distance = \", was)"
+        "was = np.sum(TP * dist_mat)\n",
+        "print(\"Your Wasserstein distance = \", was)\n",
+        "print(\"Correct answer =  0.15148578811369506\")"
       ],
       "metadata": {
         "id": "yiQ_8j-Raq3c"

Notebooks/Chap17/17_2_Reparameterization_Trick.ipynb

@@ -1,18 +1,16 @@
 {
   "cells": [
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
-        "colab_type": "text",
-        "id": "view-in-github"
+        "id": "view-in-github",
+        "colab_type": "text"
       },
       "source": [
         "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/Notebooks/Chap17/17_2_Reparameterization_Trick.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "t9vk9Elugvmi"
@@ -40,7 +38,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "paLz5RukZP1J"
@@ -114,7 +111,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "r5Hl2QkimWx9"
@@ -139,13 +135,12 @@
         "\n",
         "fig,ax = plt.subplots()\n",
         "ax.plot(phi_vals, expected_vals,'r-')\n",
-        "ax.set_xlabel('Parameter $\\phi$')\n",
-        "ax.set_ylabel('$\\mathbb{E}_{Pr(x|\\phi)}[f[x]]$')\n",
+        "ax.set_xlabel(r'Parameter $\\phi$')\n",
+        "ax.set_ylabel(r'$\\mathbb{E}_{Pr(x|\\phi)}[f[x]]$')\n",
         "plt.show()"
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "zTCykVeWqj_O"
@@ -253,13 +248,12 @@
         "\n",
         "fig,ax = plt.subplots()\n",
         "ax.plot(phi_vals, deriv_vals,'r-')\n",
-        "ax.set_xlabel('Parameter $\\phi$')\n",
-        "ax.set_ylabel('$\\partial/\\partial\\phi\\mathbb{E}_{Pr(x|\\phi)}[f[x]]$')\n",
+        "ax.set_xlabel(r'Parameter $\\phi$')\n",
+        "ax.set_ylabel(r'$\\partial/\\partial\\phi\\mathbb{E}_{Pr(x|\\phi)}[f[x]]$')\n",
         "plt.show()"
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "ASu4yKSwAEYI"
@@ -269,7 +263,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "xoFR1wifc8-b"
@@ -366,13 +359,12 @@
         "\n",
         "fig,ax = plt.subplots()\n",
         "ax.plot(phi_vals, deriv_vals,'r-')\n",
-        "ax.set_xlabel('Parameter $\\phi$')\n",
-        "ax.set_ylabel('$\\partial/\\partial\\phi\\mathbb{E}_{Pr(x|\\phi)}[f[x]]$')\n",
+        "ax.set_xlabel(r'Parameter $\\phi$')\n",
+        "ax.set_ylabel(r'$\\partial/\\partial\\phi\\mathbb{E}_{Pr(x|\\phi)}[f[x]]$')\n",
         "plt.show()"
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "1TWBiUC7bQSw"
@@ -403,7 +395,6 @@
       ]
     },
     {
-      "attachments": {},
       "cell_type": "markdown",
       "metadata": {
         "id": "d-0tntSYdKPR"
@@ -415,9 +406,8 @@
   ],
   "metadata": {
     "colab": {
-      "authorship_tag": "ABX9TyOxO2/0DTH4n4zhC97qbagY",
-      "include_colab_link": true,
-      "provenance": []
+      "provenance": [],
+      "include_colab_link": true
     },
     "kernelspec": {
       "display_name": "Python 3",

Notebooks/Chap17/17_3_Importance_Sampling.ipynb

@@ -387,7 +387,7 @@
         "def compute_expectation2b(n_samples):\n",
         "  # TODO -- complete this function\n",
         "  # 1. Draw n_samples from auxiliary distribution\n",
-        "  # 2. Compute f[y] for those samples\n",
+        "  # 2. Compute f2[y] for those samples\n",
         "  # 3. Scale the results by pr_y / q_y\n",
         "  # 4. Compute the mean of these weighted samples\n",
         "  # Replace this line\n",

Notebooks/Chap18/18_1_Diffusion_Encoder.ipynb

@@ -3,8 +3,8 @@
     {
       "cell_type": "markdown",
       "metadata": {
-        "colab_type": "text",
-        "id": "view-in-github"
+        "id": "view-in-github",
+        "colab_type": "text"
       },
       "source": [
         "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/Notebooks/Chap18/18_1_Diffusion_Encoder.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
@@ -405,11 +405,11 @@
         "\n",
         "    # TODO Write this function\n",
         "    # 1. For each x (value in x_plot_vals):\n",
-        "      # 2. Compute the mean and variance of the diffusion kernel at time t\n",
-        "      # 3. Compute pdf of this Gaussian at every x_plot_val\n",
-        "      # 4. Weight Gaussian by probability at position x and by 0.01 to componensate for bin size\n",
-        "      # 5. Accumulate weighted Gaussian in marginal at time t.\n",
-        "    # 6. Multiply result by 0.01 to compensate for bin size\n",
+        "    # 2. Compute the mean and variance of the diffusion kernel at time t\n",
+        "    # 3. Compute pdf of this Gaussian at every x_plot_val\n",
+        "    # 4. Weight Gaussian by probability at position x and by 0.01 to componensate for bin size\n",
+        "    # 5. Accumulate weighted Gaussian in marginal at time t.\n",
+        "\n",
         "    # Replace this line:\n",
         "    marginal_at_time_t = marginal_at_time_t\n",
         "\n",
@@ -454,9 +454,8 @@
   ],
   "metadata": {
     "colab": {
-      "authorship_tag": "ABX9TyMpC8kgLnXx0XQBtwNAQ4jJ",
-      "include_colab_link": true,
-      "provenance": []
+      "provenance": [],
+      "include_colab_link": true
     },
     "kernelspec": {
       "display_name": "Python 3",

Notebooks/Chap19/19_3_Monte_Carlo_Methods.ipynb

@@ -1,20 +1,4 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "authorship_tag": "ABX9TyMWjsdr5SDwyzcDftnehlNo",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
       "cell_type": "markdown",
@@ -28,6 +12,9 @@
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "t9vk9Elugvmi"
+      },
       "source": [
         "# **Notebook 19.3: Monte-Carlo methods**\n",
         "\n",
@@ -37,42 +24,49 @@
         "\n",
         "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
-        "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
-      ],
-      "metadata": {
-        "id": "t9vk9Elugvmi"
-      }
+        "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
+        "\n",
+        "Thanks to [Akshil Patel](https://www.akshilpatel.com) and [Jessica Nicholson](https://jessicanicholson1.github.io) for their help in preparing this notebook."
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "import numpy as np\n",
-        "import matplotlib.pyplot as plt\n",
-        "from PIL import Image"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "OLComQyvCIJ7"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "import numpy as np\n",
+        "import matplotlib.pyplot as plt\n",
+        "from PIL import Image\n",
+        "\n",
+        "from IPython.display import clear_output\n",
+        "from time import sleep"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ZsvrUszPLyEG"
+      },
+      "outputs": [],
       "source": [
         "# Get local copies of components of images\n",
         "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Empty.png\n",
         "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Hole.png\n",
         "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Fish.png\n",
         "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Penguin.png"
-      ],
-      "metadata": {
-        "id": "ZsvrUszPLyEG"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Gq1HfJsHN3SB"
+      },
+      "outputs": [],
       "source": [
         "# Ugly class that takes care of drawing pictures like in the book.\n",
         "# You can totally ignore this code!\n",
@@ -257,205 +251,281 @@
         "\n",
         "\n",
         "    plt.show()"
-      ],
-      "metadata": {
-        "id": "Gq1HfJsHN3SB"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "eBQ7lTpJQBSe"
+      },
+      "outputs": [],
       "source": [
         "# We're going to work on the problem depicted in figure 19.10a\n",
         "n_rows = 4; n_cols = 4\n",
         "layout = np.zeros(n_rows * n_cols)\n",
         "reward_structure = np.zeros(n_rows * n_cols)\n",
-        "layout[9] = 1 ; reward_structure[9] = -2\n",
-        "layout[10] = 1; reward_structure[10] = -2\n",
-        "layout[14] = 1; reward_structure[14] = -2\n",
-        "layout[15] = 2; reward_structure[15] = 3\n",
+        "layout[9] = 1 ; reward_structure[9] = -2    # Hole\n",
+        "layout[10] = 1; reward_structure[10] = -2   # Hole\n",
+        "layout[14] = 1; reward_structure[14] = -2   # Hole\n",
+        "layout[15] = 2; reward_structure[15] = 3    # Fish\n",
         "initial_state = 0\n",
         "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
         "mdp_drawer.draw(layout, state = initial_state, rewards=reward_structure, draw_state_index = True)"
-      ],
-      "metadata": {
-        "id": "eBQ7lTpJQBSe"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "For clarity, the black numbers are the state number and the red numbers are the reward for being in that state.  Note that the states are indexed from 0 rather than 1 as in the book to make the code neater."
-      ],
       "metadata": {
         "id": "6Vku6v_se2IG"
-      }
+      },
+      "source": [
+        "For clarity, the black numbers are the state number and the red numbers are the reward for being in that state.  Note that the states are indexed from 0 rather than 1 as in the book to make the code neater."
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "Fhc6DzZNOjiC"
+      },
       "source": [
         "Now let's define the state transition function $Pr(s_{t+1}|s_{t},a)$ in full where $a$ is the actions.  Here $a=0$ means try to go upward, $a=1$, right, $a=2$ down and $a=3$ right.  However, the ice is slippery, so we don't always go the direction we want to.\n",
         "\n",
         "Note that as for the states, we've indexed the actions from zero (unlike in the book) so they map to the indices of arrays better"
-      ],
-      "metadata": {
-        "id": "Fhc6DzZNOjiC"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "l7rT78BbOgTi"
+      },
+      "outputs": [],
       "source": [
         "transition_probabilities_given_action0 = np.array(\\\n",
-        "[[0.00 , 0.33, 0.00, 0.00,  0.50, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.50 , 0.00, 0.33, 0.00,  0.00, 0.50, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.33, 0.00, 0.50,  0.00, 0.00, 0.50, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.33, 0.00,  0.00, 0.00, 0.00, 0.50,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.50 , 0.00, 0.00, 0.00,  0.00, 0.17, 0.00, 0.00,   0.50, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.34, 0.00, 0.00,  0.25, 0.00, 0.17, 0.00,   0.00, 0.50, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.34, 0.00,  0.00, 0.17, 0.00, 0.25,   0.00, 0.00, 0.50, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.50,  0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.50,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.25, 0.00, 0.00, 0.00,   0.00, 0.17, 0.00, 0.00,   0.75, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.16, 0.00, 0.00,   0.25, 0.00, 0.17, 0.00,   0.00, 0.50, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.16, 0.00,   0.00, 0.17, 0.00, 0.25,   0.00, 0.00, 0.50, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.75 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.25, 0.00, 0.00, 0.00,   0.00, 0.25, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.16, 0.00, 0.00,   0.25, 0.00, 0.25, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.16, 0.00,   0.00, 0.25, 0.00, 0.25 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.25, 0.00 ],\n",
-        "])\n",
+        "[[0.90, 0.05, 0.00, 0.00,  0.85, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.05, 0.85, 0.05, 0.00,  0.00, 0.85, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.05, 0.85, 0.05,  0.00, 0.00, 0.85, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.05, 0.90,  0.00, 0.00, 0.00, 0.85,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.05, 0.00, 0.00, 0.00,  0.05, 0.05, 0.00, 0.00,  0.85, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.05, 0.00, 0.00,  0.05, 0.00, 0.05, 0.00,  0.00, 0.85, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.05, 0.00,  0.00, 0.05, 0.00, 0.05,  0.00, 0.00, 0.85, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.05, 0.05,  0.00, 0.00, 0.00, 0.85,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00,  0.05, 0.05, 0.00, 0.00,  0.85, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.05, 0.00, 0.00,  0.05, 0.00, 0.05, 0.00,  0.00, 0.85, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.05, 0.00,  0.00, 0.05, 0.00, 0.05,  0.00, 0.00, 0.85, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.05, 0.05,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00,  0.10, 0.05, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.05, 0.00, 0.00,  0.05, 0.05, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.05, 0.00,  0.00, 0.05, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.05, 0.00]])\n",
+        "\n",
         "\n",
         "transition_probabilities_given_action1 = np.array(\\\n",
-        "[[0.00 , 0.25, 0.00, 0.00,  0.25, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.75 , 0.00, 0.25, 0.00,  0.00, 0.17, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.50, 0.00, 0.50,  0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.50, 0.00,  0.00, 0.00, 0.00, 0.33,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.25 , 0.00, 0.00, 0.00,  0.00, 0.17, 0.00, 0.00,   0.25, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.25, 0.00, 0.00,  0.50, 0.00, 0.17, 0.00,   0.00, 0.17, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.25, 0.00,  0.00, 0.50, 0.00, 0.33,   0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.50,  0.00, 0.00, 0.50, 0.00,   0.00, 0.00, 0.00, 0.33,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.25, 0.00, 0.00, 0.00,   0.00, 0.17, 0.00, 0.00,   0.25, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.16, 0.00, 0.00,   0.50, 0.00, 0.17, 0.00,   0.00, 0.25, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.16, 0.00,   0.00, 0.50, 0.00, 0.33,   0.00, 0.00, 0.25, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.34,   0.00, 0.00, 0.50, 0.00,   0.00, 0.00, 0.00, 0.50 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.25, 0.00, 0.00, 0.00,   0.00, 0.25, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.16, 0.00, 0.00,   0.75, 0.00, 0.25, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.16, 0.00,   0.00, 0.50, 0.00, 0.50 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.34,   0.00, 0.00, 0.50, 0.00 ],\n",
-        "])\n",
+        "[[0.10, 0.05, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.85, 0.05, 0.05, 0.00,  0.00, 0.05, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.85, 0.05, 0.05,  0.00, 0.00, 0.05, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.85, 0.90,  0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.05, 0.00, 0.00, 0.00,  0.05, 0.05, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.05, 0.00, 0.00,  0.85, 0.00, 0.05, 0.00,  0.00, 0.05, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.05, 0.00,  0.00, 0.85, 0.00, 0.05,  0.00, 0.00, 0.05, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.85, 0.85,  0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00,  0.05, 0.05, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.05, 0.00, 0.00,  0.85, 0.00, 0.05, 0.00,  0.00, 0.05, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.05, 0.00,  0.00, 0.85, 0.00, 0.05,  0.00, 0.00, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.85, 0.85,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00,  0.10, 0.05, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.05, 0.00, 0.00,  0.85, 0.05, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.05, 0.00,  0.00, 0.85, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.85, 0.00]])\n",
+        "\n",
         "\n",
         "transition_probabilities_given_action2 = np.array(\\\n",
-        "[[0.00 , 0.25, 0.00, 0.00,  0.25, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.25 , 0.00, 0.25, 0.00,  0.00, 0.17, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.25, 0.00, 0.25,  0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.25, 0.00,  0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.75 , 0.00, 0.00, 0.00,  0.00, 0.17, 0.00, 0.00,   0.25, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.50, 0.00, 0.00,  0.25, 0.00, 0.17, 0.00,   0.00, 0.17, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.50, 0.00,  0.00, 0.16, 0.00, 0.25,   0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.75,  0.00, 0.00, 0.16, 0.00,   0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.50, 0.00, 0.00, 0.00,   0.00, 0.17, 0.00, 0.00,   0.50, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.50, 0.00, 0.00,   0.25, 0.00, 0.17, 0.00,   0.00, 0.33, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.50, 0.00,   0.00, 0.16, 0.00, 0.25,   0.00, 0.00, 0.33, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.50,   0.00, 0.00, 0.16, 0.00,   0.00, 0.00, 0.00, 0.50 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.50, 0.00, 0.00, 0.00,   0.00, 0.33, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.50, 0.00, 0.00,   0.50, 0.00, 0.33, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.50, 0.00,   0.00, 0.34, 0.00, 0.50 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.50,   0.00, 0.00, 0.34, 0.00 ],\n",
-        "])\n",
+        "[[0.10, 0.05, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.05, 0.05, 0.05, 0.00,  0.00, 0.05, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.05, 0.05, 0.05,  0.00, 0.00, 0.05, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.05, 0.10,  0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.85, 0.00, 0.00, 0.00,  0.05, 0.05, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.85, 0.00, 0.00,  0.05, 0.00, 0.05, 0.00,  0.00, 0.05, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.85, 0.00,  0.00, 0.05, 0.00, 0.05,  0.00, 0.00, 0.05, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.85,  0.00, 0.00, 0.05, 0.05,  0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.85, 0.00, 0.00, 0.00,  0.05, 0.05, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.85, 0.00, 0.00,  0.05, 0.00, 0.05, 0.00,  0.00, 0.05, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.85, 0.00,  0.00, 0.05, 0.00, 0.05,  0.00, 0.00, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.85,  0.00, 0.00, 0.05, 0.05,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.85, 0.00, 0.00, 0.00,  0.90, 0.05, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.85, 0.00, 0.00,  0.05, 0.85, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.85, 0.00,  0.00, 0.05, 0.85, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.85,  0.00, 0.00, 0.05, 0.00]])\n",
         "\n",
         "transition_probabilities_given_action3 = np.array(\\\n",
-        "[[0.00 , 0.25, 0.00, 0.00,  0.33, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.50 , 0.00, 0.25, 0.00,  0.00, 0.17, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.50, 0.00, 0.75,  0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.50, 0.00,  0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.50 , 0.00, 0.00, 0.00,  0.00, 0.50, 0.00, 0.00,   0.33, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.25, 0.00, 0.00,  0.33, 0.00, 0.50, 0.00,   0.00, 0.17, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.25, 0.00,  0.00, 0.17, 0.00, 0.50,   0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.25,  0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.34, 0.00, 0.00, 0.00,   0.00, 0.50, 0.00, 0.00,   0.50, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.16, 0.00, 0.00,   0.33, 0.00, 0.50, 0.00,   0.00, 0.25, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.16, 0.00,   0.00, 0.17, 0.00, 0.50,   0.00, 0.00, 0.25, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.25 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.34, 0.00, 0.00, 0.00,   0.00, 0.50, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.16, 0.00, 0.00,   0.50, 0.00, 0.50, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.16, 0.00,   0.00, 0.25, 0.00, 0.75 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.25, 0.00 ],\n",
-        "])\n",
+        "[[0.90, 0.85, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.05, 0.05, 0.85, 0.00,  0.00, 0.05, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.05, 0.05, 0.85,  0.00, 0.00, 0.05, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.05, 0.10,  0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.05, 0.00, 0.00, 0.00,  0.85, 0.85, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.05, 0.00, 0.00,  0.05, 0.00, 0.85, 0.00,  0.00, 0.05, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.05, 0.00,  0.00, 0.05, 0.00, 0.85,  0.00, 0.00, 0.05, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.05, 0.05,  0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00,  0.85, 0.85, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.05, 0.00, 0.00,  0.05, 0.00, 0.85, 0.00,  0.00, 0.05, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.05, 0.00,  0.00, 0.05, 0.00, 0.85,  0.00, 0.00, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.05, 0.05,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00,  0.90, 0.85, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.05, 0.00, 0.00,  0.05, 0.05, 0.85, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.05, 0.00,  0.00, 0.05, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.05, 0.00]])\n",
+        "\n",
+        "\n",
         "\n",
         "# Store all of these in a three dimension array\n",
         "# Pr(s_{t+1}=2|s_{t}=1, a_{t}=3] is stored at position [2,1,3]\n",
         "transition_probabilities_given_action = np.concatenate((np.expand_dims(transition_probabilities_given_action0,2),\n",
         "                                                        np.expand_dims(transition_probabilities_given_action1,2),\n",
         "                                                        np.expand_dims(transition_probabilities_given_action2,2),\n",
-        "                                                        np.expand_dims(transition_probabilities_given_action3,2)),axis=2)"
-      ],
+        "                                                        np.expand_dims(transition_probabilities_given_action3,2)),axis=2)\n",
+        "\n",
+        "print('Grid Size:', len(transition_probabilities_given_action[0]))\n",
+        "print()\n",
+        "print('Transition Probabilities for when next state = 2:')\n",
+        "print(transition_probabilities_given_action[2])\n",
+        "print()\n",
+        "print('Transitions Probabilities for when next state = 2 and current state = 1')\n",
+        "print(transition_probabilities_given_action[2][1])\n",
+        "print()\n",
+        "print('Transitions Probabilities for  when next state = 2 and current state = 1 and action = 3 (Left):')\n",
+        "print(transition_probabilities_given_action[2][1][3])"
+      ]
+    },
+    {
+      "cell_type": "markdown",
       "metadata": {
-        "id": "l7rT78BbOgTi"
+        "id": "BHWjp6Qq4tBF"
       },
-      "execution_count": null,
-      "outputs": []
+      "source": [
+        "## Implementation Details\n",
+        "\n",
+        "We provide the following methods:\n",
+        "\n",
+        "- **`markov_decision_process_step_stochastic`** - this function selects an action based on the stochastic policy for the current state, updates the state based on the transition probabilities associated with the chosen action, and returns the new state, the reward obtained for the new state, the chosen action, and whether the episode terminates.\n",
+        "\n",
+        "- **`get_one_episode`** - this function simulates an episode of agent-environment interaction. It returns the states, rewards, and actions seen in that episode, which we can then use to update the agent.\n",
+        "\n",
+        "- **`calculate_returns`** - this function calls on the **`calculate_return`** function that computes the discounted sum of rewards from a specific step, in a sequence of rewards.\n",
+        "\n",
+        "You have to implement the following methods:\n",
+        "\n",
+        "- **`deterministic_policy_to_epsilon_greedy`** - given a deterministic policy, where one action is chosen per state, this function computes the $\\epsilon$-greedy version of that policy, where each of the four actions has some nonzero probability of being selected per state. In each state, the probability of selecting each of the actions should sum to 1.\n",
+        "\n",
+        "- **`update_policy_mc`** - this function updates the action-value function using the Monte Carlo method.  We use the rollout trajectories collected using `get_one_episode` to calculate the returns. Then update the action values towards the Monte Carlo estimate of the return for each state."
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "akjrncMF-FkU"
+      },
+      "outputs": [],
       "source": [
         "# This takes a single step from an MDP\n",
-        "def markov_decision_process_step_stochastic(state, transition_probabilities_given_action, reward_structure, stochastic_policy):\n",
+        "def markov_decision_process_step_stochastic(state, transition_probabilities_given_action, reward_structure, terminal_states, stochastic_policy):\n",
         "  # Pick action\n",
         "  action = np.random.choice(a=np.arange(0,4,1),p=stochastic_policy[:,state])\n",
+        "\n",
         "  # Update the state\n",
         "  new_state = np.random.choice(a=np.arange(0,transition_probabilities_given_action.shape[0]),p = transition_probabilities_given_action[:,state,action])\n",
         "  # Return the reward\n",
         "  reward = reward_structure[new_state]\n",
+        "  is_terminal = new_state in [terminal_states]\n",
         "\n",
-        "  return new_state, reward, action"
-      ],
-      "metadata": {
-        "id": "akjrncMF-FkU"
-      },
-      "execution_count": null,
-      "outputs": []
+        "  return new_state, reward, action, is_terminal"
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "# Run one episode and return actions, rewards, returns\n",
-        "def get_one_episode(initial_state, transition_probabilities_given_action, reward_structure, stochastic_policy):\n",
-        "\n",
-        "  max_steps = 1000\n",
-        "  states = np.zeros(max_steps, dtype='uint8') ;\n",
-        "  rewards = np.zeros(max_steps) ;\n",
-        "  actions = np.zeros(max_steps, dtype='uint8') ;\n",
-        "\n",
-        "  t = 0\n",
-        "  states[t] = initial_state\n",
-        "  # While haven't reached maximum number of steps\n",
-        "  while t< max_steps:\n",
-        "    # Keep stepping through MDP\n",
-        "    states[t+1],rewards[t+1],actions[t] = markov_decision_process_step_stochastic(states[t], transition_probabilities_given_action, reward_structure, stochastic_policy)\n",
-        "    # If we reach te:rminal state then quit\n",
-        "    if states[t]==15:\n",
-        "      break;\n",
-        "    t+=1\n",
-        "\n",
-        "  states = states[:t+1]\n",
-        "  rewards = rewards[:t+1]\n",
-        "  actions = actions[:t+1]\n",
-        "\n",
-        "  return states, rewards, actions"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "bFYvF9nAloIA"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "# Run one episode and return actions, rewards, returns\n",
+        "def get_one_episode(initial_state, transition_probabilities_given_action, reward_structure, terminal_states, stochastic_policy):\n",
+        "\n",
+        "    states  = []\n",
+        "    rewards = []\n",
+        "    actions = []\n",
+        "\n",
+        "    states.append(initial_state)\n",
+        "    state = initial_state\n",
+        "\n",
+        "    is_terminal = False\n",
+        "    # While we haven't reached a terminal state\n",
+        "    while not is_terminal:\n",
+        "        # Keep stepping through MDP\n",
+        "        state, reward, action, is_terminal = markov_decision_process_step_stochastic(state,\n",
+        "                                                                                     transition_probabilities_given_action,\n",
+        "                                                                                     reward_structure,\n",
+        "                                                                                     terminal_states,\n",
+        "                                                                                     stochastic_policy)\n",
+        "        states.append(state)\n",
+        "        rewards.append(reward)\n",
+        "        actions.append(action)\n",
+        "\n",
+        "    states  = np.array(states, dtype=\"uint8\")\n",
+        "    rewards = np.array(rewards)\n",
+        "    actions = np.array(actions, dtype=\"uint8\")\n",
+        "\n",
+        "    # If the episode was terminated early, we need to compute the return differently using r_{t+1} + gamma*V(s_{t+1})\n",
+        "    return states, rewards, actions"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "qJhOrIId4tBF"
+      },
+      "outputs": [],
+      "source": [
+        "def visualize_one_episode(states, actions):\n",
+        "    # Define actions for visualization\n",
+        "    acts = ['up', 'right', 'down', 'left']\n",
+        "\n",
+        "    # Iterate over the states and actions\n",
+        "    for i in range(len(states)):\n",
+        "\n",
+        "        if i == 0:\n",
+        "            print('Starting State:', states[i])\n",
+        "\n",
+        "        elif i == len(states)-1:\n",
+        "            print('Episode Done:', states[i])\n",
+        "\n",
+        "        else:\n",
+        "            print('State', states[i-1])\n",
+        "            a = actions[i]\n",
+        "            print('Action:', acts[a])\n",
+        "            print('Next State:', states[i])\n",
+        "\n",
+        "        # Visualize the current state using the MDP drawer\n",
+        "        mdp_drawer.draw(layout, state=states[i], rewards=reward_structure, draw_state_index=True)\n",
+        "        clear_output(True)\n",
+        "\n",
+        "        # Pause for a short duration to allow observation\n",
+        "        sleep(1.5)\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "_AKwdtQQHzIK"
+      },
+      "outputs": [],
       "source": [
         "# Convert deterministic policy (1x16) to an epsilon greedy stochastic policy (4x16)\n",
-        "def deterministic_policy_to_epsilon_greedy(policy, epsilon=0.1):\n",
+        "def deterministic_policy_to_epsilon_greedy(policy, epsilon=0.2):\n",
         "  # TODO -- write this function\n",
         "  # You should wind up with a 4x16 matrix, with epsilon/3 in every position except the real policy\n",
         "  # The columns should sum to one\n",
@@ -464,27 +534,27 @@
         "\n",
         "\n",
         "  return stochastic_policy"
-      ],
-      "metadata": {
-        "id": "_AKwdtQQHzIK"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Let's try generating an episode"
-      ],
       "metadata": {
         "id": "OhVXw2Favo-w"
-      }
+      },
+      "source": [
+        "Let's try generating an episode"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "5zQ1Oh9Zvnwt"
+      },
+      "outputs": [],
       "source": [
         "# Set seed so random numbers always the same\n",
-        "np.random.seed(0)\n",
+        "np.random.seed(6)\n",
         "# Print in compact form\n",
         "np.set_printoptions(precision=3)\n",
         "\n",
@@ -494,32 +564,55 @@
         "# Convert deterministic policy to stochastic\n",
         "stochastic_policy = deterministic_policy_to_epsilon_greedy(policy)\n",
         "\n",
-        "print(\"Initial policy:\")\n",
+        "print(\"Initial Penguin Policy:\")\n",
         "print(policy)\n",
+        "print()\n",
+        "print('Stochastic Penguin Policy:')\n",
+        "print(stochastic_policy)\n",
+        "print()\n",
         "\n",
         "initial_state = 5\n",
-        "states, rewards, actions = get_one_episode(initial_state,transition_probabilities_given_action, reward_structure, stochastic_policy)"
-      ],
-      "metadata": {
-        "id": "5zQ1Oh9Zvnwt"
-      },
-      "execution_count": null,
-      "outputs": []
-    },
-    {
-      "cell_type": "markdown",
-      "source": [
-        "We'll need to calculate the returns (discounted cumulative reward) for each state action pair"
-      ],
-      "metadata": {
-        "id": "nl6rtNffwhcU"
-      }
+        "terminal_states=[15]\n",
+        "states, rewards, actions = get_one_episode(initial_state,transition_probabilities_given_action, reward_structure, terminal_states, stochastic_policy)\n",
+        "\n",
+        "print('Initial Penguin Position:')\n",
+        "mdp_drawer.draw(layout, state = initial_state, rewards=reward_structure, draw_state_index = True)\n",
+        "\n",
+        "print('Total steps to termination:', len(states))\n",
+        "print('Final Reward:', np.sum(rewards))"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "KJH-UGKk4tBF"
+      },
+      "outputs": [],
+      "source": [
+        "#this visualizes the complete episode\n",
+        "visualize_one_episode(states, actions)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "nl6rtNffwhcU"
+      },
+      "source": [
+        "We'll need to calculate the returns (discounted cumulative reward) for each state action pair"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "FxrItqGPLTq7"
+      },
+      "outputs": [],
       "source": [
         "def calculate_returns(rewards, gamma):\n",
-        "  returns = np.zeros_like(rewards)\n",
+        "  returns = np.zeros(len(rewards))\n",
         "  for c_return in range(len(returns)):\n",
         "    returns[c_return] = calculate_return(rewards[c_return:], gamma)\n",
         "  return returns\n",
@@ -529,26 +622,26 @@
         "  for i in range(len(rewards)):\n",
         "      return_val += rewards[i] * np.power(gamma, i)\n",
         "  return return_val"
-      ],
-      "metadata": {
-        "id": "FxrItqGPLTq7"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "This routine does the main work of the Monte Carlo method.  We repeatedly rollout episods, calculate the returns.  Then we figure out the average return for each state action pair, and choose the next policy as the action that has greatest state action value at each state."
-      ],
       "metadata": {
         "id": "DX1KfHRhzUOU"
-      }
+      },
+      "source": [
+        "This routine does the main work of the on-policy Monte Carlo method.  We repeatedly rollout episods, calculate the returns.  Then we figure out the average return for each state action pair, and choose the next policy as the action that has greatest state action value at each state."
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "hCghcKlOJXSM"
+      },
+      "outputs": [],
       "source": [
-        "def update_policy_mc(initial_state, transition_probabilities_given_action, reward_structure, stochastic_policy, gamma, n_rollouts=1):\n",
+        "def update_policy_mc(initial_state, transition_probabilities_given_action, reward_structure, terminal_states, stochastic_policy, gamma, n_rollouts=1):\n",
         "  # Create two matrices to store total returns for each action/state pair and the\n",
         "  # number of times we observed that action/state pair\n",
         "  n_state = transition_probabilities_given_action.shape[0]\n",
@@ -574,18 +667,18 @@
         "  state_action_values = state_action_returns_total/( state_action_count+0.00001)\n",
         "  policy = np.argmax(state_action_values, axis=0).astype(int)\n",
         "  return policy, state_action_values\n"
-      ],
-      "metadata": {
-        "id": "hCghcKlOJXSM"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "8jWhDlkaKj7Q"
+      },
+      "outputs": [],
       "source": [
         "# Set seed so random numbers always the same\n",
-        "np.random.seed(3)\n",
+        "np.random.seed(0)\n",
         "# Print in compact form\n",
         "np.set_printoptions(precision=3)\n",
         "\n",
@@ -597,32 +690,60 @@
         "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
         "mdp_drawer.draw(layout, policy = policy, rewards = reward_structure)\n",
         "\n",
-        "\n",
-        "n_policy_update = 5\n",
+        "terminal_states = [15]\n",
+        "# Track all the policies so we can visualize them later\n",
+        "all_policies = []\n",
+        "n_policy_update = 2000\n",
         "for c_policy_update in range(n_policy_update):\n",
-        "  # Convert policy to stochastic\n",
-        "  stochastic_policy = deterministic_policy_to_epsilon_greedy(policy)\n",
-        "  # Update policy by Monte Carlo method\n",
-        "  policy, state_action_values = update_policy_mc(initial_state, transition_probabilities_given_action, reward_structure, stochastic_policy, gamma, n_rollouts=1000)\n",
-        "  print(\"Updated policy\")\n",
-        "  print(policy)\n",
-        "  mdp_drawer = DrawMDP(n_rows, n_cols)\n",
-        "  mdp_drawer.draw(layout, policy = policy, rewards = reward_structure, state_action_values=state_action_values)\n"
-      ],
-      "metadata": {
-        "id": "8jWhDlkaKj7Q"
-      },
-      "execution_count": null,
-      "outputs": []
+        "    # Convert policy to stochastic\n",
+        "    stochastic_policy = deterministic_policy_to_epsilon_greedy(policy)\n",
+        "    # Update policy by Monte Carlo method\n",
+        "    policy, state_action_values = update_policy_mc(initial_state, transition_probabilities_given_action, reward_structure, terminal_states, stochastic_policy, gamma, n_rollouts=100)\n",
+        "    all_policies.append(policy)\n",
+        "\n",
+        "    # Print out 10 snapshots of progress\n",
+        "    if (c_policy_update % (n_policy_update//10) == 0) or c_policy_update == n_policy_update - 1:\n",
+        "        print(\"Updated policy\")\n",
+        "        print(policy)\n",
+        "        mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+        "        mdp_drawer.draw(layout, policy = policy, rewards = reward_structure, state_action_values=state_action_values)\n",
+        "\n",
+        "\n"
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "You can see that the results are quite noisy, but there is a definite improvement from the initial policy."
-      ],
       "metadata": {
         "id": "j7Ny47kTEMzH"
-      }
+      },
+      "source": [
+        "You can see a definite improvement to the policy"
+      ]
     }
-  ]
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3 (ipykernel)",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.10.12"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
 }

Notebooks/Chap19/19_4_Temporal_Difference_Methods.ipynb

@@ -1,20 +1,4 @@
 {
-  "nbformat": 4,
-  "nbformat_minor": 0,
-  "metadata": {
-    "colab": {
-      "provenance": [],
-      "authorship_tag": "ABX9TyNEAhORON7DFN1dZMhDK/PO",
-      "include_colab_link": true
-    },
-    "kernelspec": {
-      "name": "python3",
-      "display_name": "Python 3"
-    },
-    "language_info": {
-      "name": "python"
-    }
-  },
   "cells": [
     {
       "cell_type": "markdown",
@@ -28,6 +12,9 @@
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "t9vk9Elugvmi"
+      },
       "source": [
         "# **Notebook 19.4: Temporal difference methods**\n",
         "\n",
@@ -35,42 +22,49 @@
         "\n",
         "Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
         "\n",
-        "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
-      ],
-      "metadata": {
-        "id": "t9vk9Elugvmi"
-      }
+        "Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
+        "\n",
+        "Thanks to [Akshil Patel](https://www.akshilpatel.com) and [Jessica Nicholson](https://jessicanicholson1.github.io) for their help in preparing this notebook."
+      ]
     },
     {
       "cell_type": "code",
-      "source": [
-        "import numpy as np\n",
-        "import matplotlib.pyplot as plt\n",
-        "from PIL import Image"
-      ],
+      "execution_count": null,
       "metadata": {
         "id": "OLComQyvCIJ7"
       },
-      "execution_count": null,
-      "outputs": []
+      "outputs": [],
+      "source": [
+        "import numpy as np\n",
+        "import matplotlib.pyplot as plt\n",
+        "from PIL import Image\n",
+        "from IPython.display import clear_output\n",
+        "from time import sleep\n",
+        "from copy import deepcopy"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ZsvrUszPLyEG"
+      },
+      "outputs": [],
       "source": [
         "# Get local copies of components of images\n",
         "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Empty.png\n",
         "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Hole.png\n",
         "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Fish.png\n",
         "!wget https://raw.githubusercontent.com/udlbook/udlbook/main/Notebooks/Chap19/Penguin.png"
-      ],
-      "metadata": {
-        "id": "ZsvrUszPLyEG"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Gq1HfJsHN3SB"
+      },
+      "outputs": [],
       "source": [
         "# Ugly class that takes care of drawing pictures like in the book.\n",
         "# You can totally ignore this code!\n",
@@ -253,269 +247,516 @@
         "          self.draw_text(\"%2.2f\"%(state_action_values[3, c_cell]), np.floor(c_cell/self.n_col), c_cell-np.floor(c_cell/self.n_col)*self.n_col,'lc','black')\n",
         "\n",
         "    plt.show()"
-      ],
+      ]
+    },
+    {
+      "cell_type": "markdown",
       "metadata": {
-        "id": "Gq1HfJsHN3SB"
+        "id": "JU8gX59o76xM"
       },
-      "execution_count": null,
-      "outputs": []
+      "source": [
+        "# Penguin Ice Environment\n",
+        "\n",
+        "In this implementation we have designed an icy gridworld that a penguin has to traverse to reach the fish found in the bottom right corner.\n",
+        "\n",
+        "## Environment Description\n",
+        "\n",
+        "Consider having to cross an icy surface to reach the yummy fish. In order to achieve this task as quickly as possible, the penguin needs to waddle along as fast as it can whilst simultaneously avoiding falling into the holes.\n",
+        "\n",
+        "In this icy environment the penguin is at one of the discrete cells in the gridworld. The agent starts each episode on a randomly chosen cell. The environment state dynamics are captured by the transition probabilities $Pr(s_{t+1} |s_t, a_t)$ where $s_t$ is the current state, $a_t$ is the action chosen, and $s_{t+1}$ is the next state at decision stage t. At each decision stage, the penguin can move in one of four directions: $a=0$ means try to go upward, $a=1$, right, $a=2$ down and $a=3$ left.\n",
+        "\n",
+        "However, the ice is slippery, so we don't always go the direction we want to: every time the agent chooses an action, with 0.25 probability, the environment changes the action taken to a differenct action, which is uniformly sampled from the other available actions.\n",
+        "\n",
+        "The rewards are deterministic; the penguin will receive a reward of +3 if it reaches the fish, -2 if it slips into a hole and 0 otherwise.\n",
+        "\n",
+        "Note that as for the states, we've indexed the actions from zero (unlike in the book) so they map to the indices of arrays better"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "eBQ7lTpJQBSe"
+      },
+      "outputs": [],
       "source": [
         "# We're going to work on the problem depicted in figure 19.10a\n",
         "n_rows = 4; n_cols = 4\n",
         "layout = np.zeros(n_rows * n_cols)\n",
         "reward_structure = np.zeros(n_rows * n_cols)\n",
-        "layout[9] = 1 ; reward_structure[9] = -2\n",
-        "layout[10] = 1; reward_structure[10] = -2\n",
-        "layout[14] = 1; reward_structure[14] = -2\n",
-        "layout[15] = 2; reward_structure[15] = 3\n",
+        "layout[9] = 1 ; reward_structure[9] = -2    # Hole\n",
+        "layout[10] = 1; reward_structure[10] = -2   # Hole\n",
+        "layout[14] = 1; reward_structure[14] = -2   # Hole\n",
+        "layout[15] = 2; reward_structure[15] = 3    # Fish\n",
         "initial_state = 0\n",
         "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
         "mdp_drawer.draw(layout, state = initial_state, rewards=reward_structure, draw_state_index = True)"
-      ],
-      "metadata": {
-        "id": "eBQ7lTpJQBSe"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "For clarity, the black numbers are the state number and the red numbers are the reward for being in that state.  Note that the states are indexed from 0 rather than 1 as in the book to make the code neater."
-      ],
       "metadata": {
         "id": "6Vku6v_se2IG"
-      }
+      },
+      "source": [
+        "For clarity, the black numbers are the state number and the red numbers are the reward for being in that state.  Note that the states are indexed from 0 rather than 1 as in the book to make the code neater."
+      ]
     },
     {
       "cell_type": "markdown",
+      "metadata": {
+        "id": "Fhc6DzZNOjiC"
+      },
       "source": [
         "Now let's define the state transition function $Pr(s_{t+1}|s_{t},a)$ in full where $a$ is the actions.  Here $a=0$ means try to go upward, $a=1$, right, $a=2$ down and $a=3$ right.  However, the ice is slippery, so we don't always go the direction we want to.\n",
         "\n",
         "Note that as for the states, we've indexed the actions from zero (unlike in the book) so they map to the indices of arrays better"
-      ],
-      "metadata": {
-        "id": "Fhc6DzZNOjiC"
-      }
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "wROjgnqh76xN"
+      },
+      "outputs": [],
       "source": [
         "transition_probabilities_given_action0 = np.array(\\\n",
-        "[[0.00 , 0.33, 0.00, 0.00,  0.50, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.50 , 0.00, 0.33, 0.00,  0.00, 0.50, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.33, 0.00, 0.50,  0.00, 0.00, 0.50, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.33, 0.00,  0.00, 0.00, 0.00, 0.50,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.50 , 0.00, 0.00, 0.00,  0.00, 0.17, 0.00, 0.00,   0.50, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.34, 0.00, 0.00,  0.25, 0.00, 0.17, 0.00,   0.00, 0.50, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.34, 0.00,  0.00, 0.17, 0.00, 0.25,   0.00, 0.00, 0.50, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.50,  0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.50,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.25, 0.00, 0.00, 0.00,   0.00, 0.17, 0.00, 0.00,   0.75, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.16, 0.00, 0.00,   0.25, 0.00, 0.17, 0.00,   0.00, 0.50, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.16, 0.00,   0.00, 0.17, 0.00, 0.25,   0.00, 0.00, 0.50, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.75 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.25, 0.00, 0.00, 0.00,   0.00, 0.25, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.16, 0.00, 0.00,   0.25, 0.00, 0.25, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.16, 0.00,   0.00, 0.25, 0.00, 0.25 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.25, 0.00 ],\n",
-        "])\n",
+        "[[0.90, 0.05, 0.00, 0.00,  0.85, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.05, 0.85, 0.05, 0.00,  0.00, 0.85, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.05, 0.85, 0.05,  0.00, 0.00, 0.85, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.05, 0.90,  0.00, 0.00, 0.00, 0.85,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.05, 0.00, 0.00, 0.00,  0.05, 0.05, 0.00, 0.00,  0.85, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.05, 0.00, 0.00,  0.05, 0.00, 0.05, 0.00,  0.00, 0.85, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.05, 0.00,  0.00, 0.05, 0.00, 0.05,  0.00, 0.00, 0.85, 0.00,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.05, 0.05,  0.00, 0.00, 0.00, 0.85,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00,  0.05, 0.05, 0.00, 0.00,  0.85, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.05, 0.00, 0.00,  0.05, 0.00, 0.05, 0.00,  0.00, 0.85, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.05, 0.00,  0.00, 0.05, 0.00, 0.05,  0.00, 0.00, 0.85, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.05, 0.05,  0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.05, 0.00, 0.00, 0.00,  0.10, 0.05, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.05, 0.00, 0.00,  0.05, 0.05, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.05, 0.00,  0.00, 0.05, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.05,  0.00, 0.00, 0.05, 0.00]])\n",
+        "\n",
         "\n",
         "transition_probabilities_given_action1 = np.array(\\\n",
-        "[[0.00 , 0.25, 0.00, 0.00,  0.25, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.75 , 0.00, 0.25, 0.00,  0.00, 0.17, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.50, 0.00, 0.50,  0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.50, 0.00,  0.00, 0.00, 0.00, 0.33,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.25 , 0.00, 0.00, 0.00,  0.00, 0.17, 0.00, 0.00,   0.25, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.25, 0.00, 0.00,  0.50, 0.00, 0.17, 0.00,   0.00, 0.17, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.25, 0.00,  0.00, 0.50, 0.00, 0.33,   0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.50,  0.00, 0.00, 0.50, 0.00,   0.00, 0.00, 0.00, 0.33,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.25, 0.00, 0.00, 0.00,   0.00, 0.17, 0.00, 0.00,   0.25, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.16, 0.00, 0.00,   0.50, 0.00, 0.17, 0.00,   0.00, 0.25, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.16, 0.00,   0.00, 0.50, 0.00, 0.33,   0.00, 0.00, 0.25, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.34,   0.00, 0.00, 0.50, 0.00,   0.00, 0.00, 0.00, 0.50 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.25, 0.00, 0.00, 0.00,   0.00, 0.25, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.16, 0.00, 0.00,   0.75, 0.00, 0.25, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.16, 0.00,   0.00, 0.50, 0.00, 0.50 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.34,   0.00, 0.00, 0.50, 0.00 ],\n",
-        "])\n",
+        "[[0.10, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.85, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.85, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.85, 0.90, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.05, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.85, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.85, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.10, 0.05, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.05, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.85, 0.00]])\n",
+        "\n",
         "\n",
         "transition_probabilities_given_action2 = np.array(\\\n",
-        "[[0.00 , 0.25, 0.00, 0.00,  0.25, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.25 , 0.00, 0.25, 0.00,  0.00, 0.17, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.25, 0.00, 0.25,  0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.25, 0.00,  0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.75 , 0.00, 0.00, 0.00,  0.00, 0.17, 0.00, 0.00,   0.25, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.50, 0.00, 0.00,  0.25, 0.00, 0.17, 0.00,   0.00, 0.17, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.50, 0.00,  0.00, 0.16, 0.00, 0.25,   0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.75,  0.00, 0.00, 0.16, 0.00,   0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.50, 0.00, 0.00, 0.00,   0.00, 0.17, 0.00, 0.00,   0.50, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.50, 0.00, 0.00,   0.25, 0.00, 0.17, 0.00,   0.00, 0.33, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.50, 0.00,   0.00, 0.16, 0.00, 0.25,   0.00, 0.00, 0.33, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.50,   0.00, 0.00, 0.16, 0.00,   0.00, 0.00, 0.00, 0.50 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.50, 0.00, 0.00, 0.00,   0.00, 0.33, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.50, 0.00, 0.00,   0.50, 0.00, 0.33, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.50, 0.00,   0.00, 0.34, 0.00, 0.50 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.50,   0.00, 0.00, 0.34, 0.00 ],\n",
-        "])\n",
+        "[[0.10, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.05, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.05, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.05, 0.10, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.85, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.00, 0.90, 0.05, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.85, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.85, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00]])\n",
         "\n",
         "transition_probabilities_given_action3 = np.array(\\\n",
-        "[[0.00 , 0.25, 0.00, 0.00,  0.33, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.50 , 0.00, 0.25, 0.00,  0.00, 0.17, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.50, 0.00, 0.75,  0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.50, 0.00,  0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.50 , 0.00, 0.00, 0.00,  0.00, 0.50, 0.00, 0.00,   0.33, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.25, 0.00, 0.00,  0.33, 0.00, 0.50, 0.00,   0.00, 0.17, 0.00, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.25, 0.00,  0.00, 0.17, 0.00, 0.50,   0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.25,  0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.34, 0.00, 0.00, 0.00,   0.00, 0.50, 0.00, 0.00,   0.50, 0.00, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.16, 0.00, 0.00,   0.33, 0.00, 0.50, 0.00,   0.00, 0.25, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.16, 0.00,   0.00, 0.17, 0.00, 0.50,   0.00, 0.00, 0.25, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.17, 0.00,   0.00, 0.00, 0.00, 0.25 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.34, 0.00, 0.00, 0.00,   0.00, 0.50, 0.00, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.16, 0.00, 0.00,   0.50, 0.00, 0.50, 0.00 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.16, 0.00,   0.00, 0.25, 0.00, 0.75 ],\n",
-        " [0.00 , 0.00, 0.00, 0.00,  0.00, 0.00, 0.00, 0.00,   0.00, 0.00, 0.00, 0.25,   0.00, 0.00, 0.25, 0.00 ],\n",
-        "])\n",
+        "[[0.90, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.05, 0.05, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.05, 0.05, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.05, 0.10, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.05, 0.00, 0.00, 0.00, 0.85, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.85, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00, 0.85, 0.00, 0.00, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.00, 0.00, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.00, 0.90, 0.85, 0.00, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.85, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.05, 0.00],\n",
+        " [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.05, 0.00, 0.00, 0.05, 0.00]])\n",
+        "\n",
+        "\n",
         "\n",
         "# Store all of these in a three dimension array\n",
         "# Pr(s_{t+1}=2|s_{t}=1, a_{t}=3] is stored at position [2,1,3]\n",
         "transition_probabilities_given_action = np.concatenate((np.expand_dims(transition_probabilities_given_action0,2),\n",
         "                                                        np.expand_dims(transition_probabilities_given_action1,2),\n",
         "                                                        np.expand_dims(transition_probabilities_given_action2,2),\n",
-        "                                                        np.expand_dims(transition_probabilities_given_action3,2)),axis=2)"
-      ],
+        "                                                        np.expand_dims(transition_probabilities_given_action3,2)),axis=2)\n",
+        "\n",
+        "print('Grid Size:', len(transition_probabilities_given_action[0]))\n",
+        "print()\n",
+        "print('Transition Probabilities for when next state = 2:')\n",
+        "print(transition_probabilities_given_action[2])\n",
+        "print()\n",
+        "print('Transitions Probabilities for when next state = 2 and current state = 1')\n",
+        "print(transition_probabilities_given_action[2][1])\n",
+        "print()\n",
+        "print('Transitions Probabilities for  when next state = 2 and current state = 1 and action = 3 (Left):')\n",
+        "print(transition_probabilities_given_action[2][1][3])"
+      ]
+    },
+    {
+      "cell_type": "markdown",
       "metadata": {
-        "id": "l7rT78BbOgTi"
+        "id": "eblSQ6xZ76xN"
       },
-      "execution_count": null,
-      "outputs": []
+      "source": [
+        "## Implementation Details\n",
+        "\n",
+        "We provide the following methods:\n",
+        "- **`markov_decision_process_step`** - this function simulates $Pr(s_{t+1} | s_{t}, a_{t})$. It randomly selects an action, updates the state based on the transition probabilities associated with the chosen action, and returns the new state, the reward obtained for leaving the current state, and the chosen action. The randomness in action selection and state transitions reflects a random exploration process and the stochastic nature of the MDP, respectively.\n",
+        "\n",
+        "- **`get_policy`** - this function computes a policy that acts greedily with respect to the state-action values. The policy is computed for all states and the action that maximizes the state-action value is chosen for each state. When there are multiple optimal actions, one is chosen at random.\n",
+        "\n",
+        "\n",
+        "You have to implement the following method:\n",
+        "\n",
+        "- **`q_learning_step`** - this function implements a single step of the Q-learning algorithm for reinforcement learning as shown below. The update follows the Q-learning formula and is controlled by parameters such as the learning rate (alpha) and the discount factor $(\\gamma)$. The function returns the updated state-action values matrix.\n",
+        "\n",
+        "$Q(s, a) \\leftarrow (1 - \\alpha) \\cdot Q(s, a) + \\alpha \\cdot \\left(r + \\gamma \\cdot \\max_{a'} Q(s', a')\\right)$"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "cKLn4Iam76xN"
+      },
+      "outputs": [],
       "source": [
-        "def q_learning_step(state_action_values, reward, state, new_state, action, gamma, alpha = 0.1):\n",
+        "def get_policy(state_action_values):\n",
+        "    policy = np.zeros(state_action_values.shape[1]) # One action for each state\n",
+        "    for state in range(state_action_values.shape[1]):\n",
+        "        # Break ties for maximising actions randomly\n",
+        "        policy[state] = np.random.choice(np.flatnonzero(state_action_values[:, state] == max(state_action_values[:, state])))\n",
+        "    return policy"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "akjrncMF-FkU"
+      },
+      "outputs": [],
+      "source": [
+        "def markov_decision_process_step(state, transition_probabilities_given_action, reward_structure, terminal_states, action=None):\n",
+        "  # Pick action\n",
+        "  if action is None:\n",
+        "    action = np.random.randint(4)\n",
+        "  # Update the state\n",
+        "  new_state = np.random.choice(a=range(transition_probabilities_given_action.shape[0]), p = transition_probabilities_given_action[:, state,action])\n",
+        "\n",
+        "  # Return the reward -- here the reward is for arriving at the state\n",
+        "  reward = reward_structure[new_state]\n",
+        "  is_terminal = new_state in [terminal_states]\n",
+        "\n",
+        "  return new_state, reward, action, is_terminal"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "5pO6-9ACWhiV"
+      },
+      "outputs": [],
+      "source": [
+        "def q_learning_step(state_action_values, reward, state, new_state, action, is_terminal, gamma, alpha = 0.1):\n",
         "  # TODO -- write this function\n",
         "  # Replace this line\n",
         "  state_action_values_after = np.copy(state_action_values)\n",
         "\n",
         "  return state_action_values_after"
-      ],
-      "metadata": {
-        "id": "5pO6-9ACWhiV"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
-      "cell_type": "code",
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "u4OHTTk176xO"
+      },
       "source": [
-        "# This takes a single step from an MDP which just has a completely random policy\n",
-        "def markov_decision_process_step(state, transition_probabilities_given_action, reward_structure):\n",
-        "  # Pick action\n",
-        "  action = np.random.randint(4)\n",
-        "  # Update the state\n",
-        "  new_state = np.random.choice(a=np.arange(0,transition_probabilities_given_action.shape[0]),p = transition_probabilities_given_action[:,state,action])\n",
-        "  # Return the reward -- here the reward is for leaving the state\n",
-        "  reward = reward_structure[state]\n",
-        "\n",
-        "  return new_state, reward, action"
-      ],
-      "metadata": {
-        "id": "akjrncMF-FkU"
-      },
-      "execution_count": null,
-      "outputs": []
+        "Lets run this for a single Q-learning step"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Fu5_VjvbSwfJ"
+      },
+      "outputs": [],
       "source": [
         "# Initialize the state-action values to random numbers\n",
         "np.random.seed(0)\n",
         "n_state = transition_probabilities_given_action.shape[0]\n",
         "n_action = transition_probabilities_given_action.shape[2]\n",
+        "terminal_states=[15]\n",
         "state_action_values = np.random.normal(size=(n_action, n_state))\n",
+        "# Hard code value of termination state of finding fish to 0\n",
+        "state_action_values[:, terminal_states] = 0\n",
         "gamma = 0.9\n",
         "\n",
-        "policy = np.argmax(state_action_values, axis=0).astype(int)\n",
+        "policy = get_policy(state_action_values)\n",
         "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
         "mdp_drawer.draw(layout, policy = policy, state_action_values = state_action_values, rewards = reward_structure)\n",
         "\n",
         "# Now let's simulate a single Q-learning step\n",
         "initial_state = 9\n",
-        "print(\"Initial state = \", initial_state)\n",
-        "new_state, reward, action = markov_decision_process_step(initial_state, transition_probabilities_given_action, reward_structure)\n",
-        "print(\"Action = \", action)\n",
-        "print(\"New state = \", new_state)\n",
-        "print(\"Reward = \", reward)\n",
+        "print(\"Initial state =\",initial_state)\n",
+        "new_state, reward, action, is_terminal = markov_decision_process_step(initial_state, transition_probabilities_given_action, reward_structure, terminal_states)\n",
+        "print(\"Action =\",action)\n",
+        "print(\"New state =\",new_state)\n",
+        "print(\"Reward =\", reward)\n",
         "\n",
-        "state_action_values_after = q_learning_step(state_action_values, reward, initial_state, new_state, action, gamma)\n",
+        "state_action_values_after = q_learning_step(state_action_values, reward, initial_state, new_state, action, is_terminal, gamma)\n",
         "print(\"Your value:\",state_action_values_after[action, initial_state])\n",
-        "print(\"True value:  0.27650262412468796\")\n",
+        "print(\"True value: 0.3024718977397814\")\n",
         "\n",
-        "policy = np.argmax(state_action_values, axis=0).astype(int)\n",
+        "policy = get_policy(state_action_values)\n",
         "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
         "mdp_drawer.draw(layout, policy = policy, state_action_values = state_action_values_after, rewards = reward_structure)\n"
-      ],
-      "metadata": {
-        "id": "Fu5_VjvbSwfJ"
-      },
-      "execution_count": null,
-      "outputs": []
+      ]
     },
     {
       "cell_type": "markdown",
-      "source": [
-        "Now let's run this for a while and watch the policy improve"
-      ],
       "metadata": {
         "id": "Ogh0qucmb68J"
-      }
+      },
+      "source": [
+        "Now let's run this for a while (20000) steps and watch the policy improve"
+      ]
     },
     {
       "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "N6gFYifh76xO"
+      },
+      "outputs": [],
       "source": [
         "# Initialize the state-action values to random numbers\n",
         "np.random.seed(0)\n",
-        "n_state = transition_probabilities_given_action.shape[0]\n",
+        "n_state  = transition_probabilities_given_action.shape[0]\n",
         "n_action = transition_probabilities_given_action.shape[2]\n",
         "state_action_values = np.random.normal(size=(n_action, n_state))\n",
-        "# Hard code termination state of finding fish\n",
-        "state_action_values[:,n_state-1] = 3.0\n",
+        "\n",
+        "# Hard code value of termination state of finding fish to 0\n",
+        "terminal_states = [15]\n",
+        "state_action_values[:, terminal_states] = 0\n",
         "gamma = 0.9\n",
         "\n",
         "# Draw the initial setup\n",
-        "policy = np.argmax(state_action_values, axis=0).astype(int)\n",
+        "print('Initial Policy:')\n",
+        "policy = get_policy(state_action_values)\n",
         "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
         "mdp_drawer.draw(layout, policy = policy, state_action_values = state_action_values, rewards = reward_structure)\n",
         "\n",
-        "\n",
-        "state= np.random.randint(n_state-1)\n",
+        "state = np.random.randint(n_state-1)\n",
         "\n",
         "# Run for a number of iterations\n",
-        "for c_iter in range(10000):\n",
-        "  new_state, reward, action = markov_decision_process_step(state, transition_probabilities_given_action, reward_structure)\n",
-        "  state_action_values_after = q_learning_step(state_action_values, reward, state, new_state, action, gamma)\n",
+        "for c_iter in range(20000):\n",
+        "  new_state, reward, action, is_terminal = markov_decision_process_step(state, transition_probabilities_given_action, reward_structure, terminal_states)\n",
+        "  state_action_values_after = q_learning_step(state_action_values, reward, state, new_state, action, is_terminal, gamma)\n",
+        "\n",
         "  # If in termination state, reset state randomly\n",
-        "  if new_state==15:\n",
-        "    state= np.random.randint(n_state-1)\n",
+        "  if is_terminal:\n",
+        "    state = np.random.randint(n_state-1)\n",
         "  else:\n",
         "    state = new_state\n",
-        "  # Update the policy\n",
-        "  state_action_values = np.copy(state_action_values_after)\n",
-        "  policy = np.argmax(state_action_values, axis=0).astype(int)\n",
         "\n",
+        "  # Update the policy\n",
+        "  state_action_values = deepcopy(state_action_values_after)\n",
+        "  policy = get_policy(state_action_values_after)\n",
+        "\n",
+        "print('Final Optimal Policy:')\n",
         "# Draw the final situation\n",
         "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
         "mdp_drawer.draw(layout, policy = policy, state_action_values = state_action_values, rewards = reward_structure)"
-      ],
+      ]
+    },
+    {
+      "cell_type": "markdown",
       "metadata": {
-        "id": "qQFhwVqPcCFH"
+        "id": "djPTKuDk76xO"
       },
+      "source": [
+        "Finally, lets run this for a **single** episode and visualize the penguin's actions"
+      ]
+    },
+    {
+      "cell_type": "code",
       "execution_count": null,
-      "outputs": []
+      "metadata": {
+        "id": "pWObQf2h76xO"
+      },
+      "outputs": [],
+      "source": [
+        "def get_one_episode(n_state, state_action_values, terminal_states, gamma):\n",
+        "\n",
+        "    state = np.random.randint(n_state-1)\n",
+        "\n",
+        "    # Create lists to store all the states seen and actions taken throughout the single episode\n",
+        "    all_states  = []\n",
+        "    all_actions = []\n",
+        "\n",
+        "    # Initalize episode termination flag\n",
+        "    done  = False\n",
+        "    # Initialize counter for steps in the episode\n",
+        "    steps = 0\n",
+        "\n",
+        "    all_states.append(state)\n",
+        "\n",
+        "    while not done:\n",
+        "      steps += 1\n",
+        "\n",
+        "      new_state, reward, action, is_terminal = markov_decision_process_step(state, transition_probabilities_given_action, reward_structure, terminal_states)\n",
+        "      all_states.append(new_state)\n",
+        "      all_actions.append(action)\n",
+        "\n",
+        "      state_action_values_after = q_learning_step(state_action_values, reward, state, new_state, action, is_terminal, gamma)\n",
+        "\n",
+        "      # If in termination state, reset state randomly\n",
+        "      if is_terminal:\n",
+        "        state = np.random.randint(n_state-1)\n",
+        "        print(f'Episode Terminated at {steps} Steps')\n",
+        "        # Set episode termination flag\n",
+        "        done = True\n",
+        "      else:\n",
+        "        state = new_state\n",
+        "\n",
+        "      # Update the policy\n",
+        "      state_action_values = deepcopy(state_action_values_after)\n",
+        "      policy = get_policy(state_action_values_after)\n",
+        "\n",
+        "    return all_states, all_actions, policy, state_action_values\n",
+        ""
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "P7cbCGT176xO"
+      },
+      "outputs": [],
+      "source": [
+        "def visualize_one_episode(states, actions):\n",
+        "    # Define actions for visualization\n",
+        "    acts = ['up', 'right', 'down', 'left']\n",
+        "\n",
+        "    # Iterate over the states and actions\n",
+        "    for i in range(len(states)):\n",
+        "\n",
+        "        if i == 0:\n",
+        "            print('Starting State:', states[i])\n",
+        "\n",
+        "        elif i == len(states)-1:\n",
+        "            print('Episode Done:', states[i])\n",
+        "\n",
+        "        else:\n",
+        "            print('State', states[i-1])\n",
+        "            a = actions[i]\n",
+        "            print('Action:', acts[a])\n",
+        "            print('Next State:', states[i])\n",
+        "\n",
+        "        # Visualize the current state using the MDP drawer\n",
+        "        mdp_drawer.draw(layout, state=states[i], rewards=reward_structure, draw_state_index=True)\n",
+        "        clear_output(True)\n",
+        "\n",
+        "        # Pause for a short duration to allow observation\n",
+        "        sleep(1.5)\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "cr98F8PT76xP"
+      },
+      "outputs": [],
+      "source": [
+        "# Initialize the state-action values to random numbers\n",
+        "np.random.seed(2)\n",
+        "n_state  = transition_probabilities_given_action.shape[0]\n",
+        "n_action = transition_probabilities_given_action.shape[2]\n",
+        "state_action_values = np.random.normal(size=(n_action, n_state))\n",
+        "\n",
+        "# Hard code value of termination state of finding fish to 0\n",
+        "terminal_states = [15]\n",
+        "state_action_values[:, terminal_states] = 0\n",
+        "gamma = 0.9\n",
+        "\n",
+        "# Draw the initial setup\n",
+        "print('Initial Policy:')\n",
+        "policy = get_policy(state_action_values)\n",
+        "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+        "mdp_drawer.draw(layout, policy = policy, state_action_values = state_action_values, rewards = reward_structure)\n",
+        "\n",
+        "states, actions, policy, state_action_values = get_one_episode(n_state, state_action_values, terminal_states, gamma)\n",
+        "\n",
+        "print()\n",
+        "print('Final Optimal Policy:')\n",
+        "mdp_drawer = DrawMDP(n_rows, n_cols)\n",
+        "mdp_drawer.draw(layout, policy = policy, state_action_values = state_action_values, rewards = reward_structure)\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "5zBu1g3776xP"
+      },
+      "outputs": [],
+      "source": [
+        "visualize_one_episode(states, actions)"
+      ]
     }
-  ]
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3 (ipykernel)",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.10.12"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
 }

Notebooks/Chap20/20_1_Random_Data.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyPkSYbEjOcEmLt8tU6HxNuR",
+      "authorship_tag": "ABX9TyNgBRvfIlngVobKuLE6leM+",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -45,8 +45,8 @@
     {
       "cell_type": "code",
       "source": [
-        "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
-        "!git clone https://github.com/greydanus/mnist1d"
+        "# Run this if you're in a Colab to install MNIST 1D repository\n",
+        "!pip install git+https://github.com/greydanus/mnist1d"
       ],
       "metadata": {
         "id": "D5yLObtZCi9J"

Notebooks/Chap20/20_2_Full_Batch_Gradient_Descent.ipynb

@@ -4,7 +4,7 @@
   "metadata": {
     "colab": {
       "provenance": [],
-      "authorship_tag": "ABX9TyOo4vm4MXcIvAzVlMCaLikH",
+      "authorship_tag": "ABX9TyO6xuszaG4nNAcWy/3juLkn",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -44,8 +44,8 @@
     {
       "cell_type": "code",
       "source": [
-        "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
-        "!git clone https://github.com/greydanus/mnist1d"
+        "# Run this if you're in a Colab to install MNIST 1D repository\n",
+        "!pip install git+https://github.com/greydanus/mnist1d"
       ],
       "metadata": {
         "id": "D5yLObtZCi9J"

Notebooks/Chap20/20_2_Full_Batch_Gradient_Descent_GPU.ipynb

@@ -5,7 +5,7 @@
     "colab": {
       "provenance": [],
       "gpuType": "T4",
-      "authorship_tag": "ABX9TyMjPBfDONmjqTSyEQDP2gjY",
+      "authorship_tag": "ABX9TyOG/5A+P053/x1IfFg52z4V",
       "include_colab_link": true
     },
     "kernelspec": {
@@ -47,8 +47,8 @@
     {
       "cell_type": "code",
       "source": [
-        "# Run this if you're in a Colab to make a local copy of the MNIST 1D repository\n",
-        "!git clone https://github.com/greydanus/mnist1d"
+        "# Run this if you're in a Colab to install MNIST 1D repository\n",
+        "!pip install git+https://github.com/greydanus/mnist1d"
       ],
       "metadata": {
         "id": "D5yLObtZCi9J"

Notebooks/Chap20/20_3_Lottery_Tickets.ipynb

@@ -43,8 +43,8 @@
         "id": "Sg2i1QmhKW5d"
       },
       "source": [
-        "# Run this if you're in a Colab\n",
-        "!git clone https://github.com/greydanus/mnist1d"
+        "# Run this if you're in a Colab to install MNIST 1D repository\n",
+        "!pip install git+https://github.com/greydanus/mnist1d"
       ],
       "execution_count": null,
       "outputs": []

index.html

@@ -1,406 +0,0 @@
-<!DOCTYPE html>
-<html lang="en">
-<head>
-    <meta charset="UTF-8">
-    <title>udlbook</title>
-    <link rel="stylesheet" href="style.css">
-</head>
-
-<body>
-<div id="head">
-    <div>
-        <h1 style="margin: 0; font-size: 36px">Understanding Deep Learning</h1>
-        by Simon J.D. Prince
-        <br>Published by MIT Press Dec 5th 2023.<br>
-        <ul>
-            <li>
-                <p style="font-size: larger; margin-bottom: 0">Download full PDF <a
-                        href="https://github.com/udlbook/udlbook/releases/download/v2.00/UnderstandingDeepLearning_28_01_24_C.pdf">here</a>
-                </p>2024-01-28. CC-BY-NC-ND license<br>
-                <img src="https://img.shields.io/github/downloads/udlbook/udlbook/total" alt="download stats shield">
-            </li>
-            <li> Order your copy from  <a href="https://mitpress.mit.edu/9780262048644/understanding-deep-learning/">here </a></li>
-            <li> Known errata can be found here:   <a
-            href="https://github.com/udlbook/udlbook/raw/main/UDL_Errata.pdf">PDF</a></li>
-            <li> Report new errata via <a href="https://github.com/udlbook/udlbook/issues">github</a>
-                or contact me directly at udlbookmail@gmail.com
-            <li> Follow me on <a href="https://twitter.com/SimonPrinceAI">Twitter</a> or <a
-                    href="https://www.linkedin.com/in/simon-prince-615bb9165/">LinkedIn</a> for updates.
-        </ul>
-        <h2>Table of contents</h2>
-        <ul>
-            <li> Chapter 1 - Introduction
-            <li> Chapter 2 - Supervised learning
-            <li> Chapter 3 - Shallow neural networks
-            <li> Chapter 4 - Deep neural networks
-            <li> Chapter 5 - Loss functions
-            <li> Chapter 6 - Training models
-            <li> Chapter 7 - Gradients and initialization
-            <li> Chapter 8 - Measuring performance
-            <li> Chapter 9 - Regularization
-            <li> Chapter 10 - Convolutional networks
-            <li> Chapter 11 - Residual networks
-            <li> Chapter 12 - Transformers
-            <li> Chapter 13 - Graph neural networks
-            <li> Chapter 14 - Unsupervised learning
-            <li> Chapter 15 - Generative adversarial networks
-            <li> Chapter 16 - Normalizing flows
-            <li> Chapter 17 - Variational autoencoders
-            <li> Chapter 18 - Diffusion models
-            <li> Chapter 19 - Deep reinforcement learning
-            <li> Chapter 20 - Why does deep learning work?
-            <li> Chapter 21 - Deep learning and ethics
-        </ul>
-    </div>
-    <div id="cover">
-        <img src="https://raw.githubusercontent.com/udlbook/udlbook/main/UDLCoverSmall.jpg"
-             alt="front cover">
-    </div>
-</div>
-<div id="body">
-    <h2>Resources for instructors </h2>
-    <p>Instructor answer booklet available with proof of credentials via <a
-            href="https://mitpress.mit.edu/9780262048644/understanding-deep-learning"> MIT Press</a>.</p>
-    <p>Request an exam/desk copy via <a href="https://mitpress.ublish.com/request?cri=15055">MIT Press</a>.</p>
-    <p>Figures in PDF (vector) / SVG (vector) / Powerpoint (images):
-    <ul>
-        <li> Chapter 1 - Introduction: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap1PDF.zip">PDF
-            Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1udnl5pUOAc8DcAQ7HQwyzP9pwL95ynnv">
-            SVG
-            Figures</a> / <a
-                href="https://docs.google.com/presentation/d/1IjTqIUvWCJc71b5vEJYte-Dwujcp7rvG/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
-            Figures</a>
-        <li> Chapter 2 - Supervised learning: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap2PDF.zip">PDF Figures</a> / <a
-                href="https://drive.google.com/uc?export=download&id=1VSxcU5y1qNFlmd3Lb3uOWyzILuOj1Dla"> SVG Figures</a>
-            /
-            <a href="https://docs.google.com/presentation/d/1Br7R01ROtRWPlNhC_KOommeHAWMBpWtz/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
-                Figures</a>
-        <li> Chapter 3 - Shallow neural networks: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap3PDF.zip">PDF Figures</a> / <a
-                href="https://drive.google.com/uc?export=download&id=19kZFWlXhzN82Zx02ByMmSZOO4T41fmqI"> SVG Figures</a>
-            /
-            <a href="https://docs.google.com/presentation/d/1e9M3jB5I9qZ4dCBY90Q3Hwft_i068QVQ/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
-                Figures</a>
-        <li> Chapter 4 - Deep neural networks: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap4PDF.zip">PDF Figures</a> / <a
-                href="https://drive.google.com/uc?export=download&id=1ojr0ebsOhzvS04ItAflX2cVmYqHQHZUa"> SVG Figures</a>
-            /
-            <a href="https://docs.google.com/presentation/d/1LTSsmY4mMrJbqXVvoTOCkQwHrRKoYnJj/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
-                Figures</a>
-        <li> Chapter 5 - Loss functions: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap5PDF.zip">PDF
-            Figures</a> / <a href="https://drive.google.com/uc?export=download&id=17MJO7fiMpFZVqKeqXTbQ36AMpmR4GizZ">
-            SVG
-            Figures</a> / <a
-                href="https://docs.google.com/presentation/d/1gcpC_3z9oRp87eMkoco-kdLD-MM54Puk/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
-            Figures</a>
-        <li> Chapter 6 - Training models: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap6PDF.zip">PDF
-            Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1VPdhFRnCr9_idTrX0UdHKGAw2shUuwhK">
-            SVG
-            Figures</a> / <a
-                href="https://docs.google.com/presentation/d/1AKoeggAFBl9yLC7X5tushAGzCCxmB7EY/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
-            Figures</a>
-        <li> Chapter 7 - Gradients and initialization: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap7PDF.zip">PDF Figures</a> / <a
-                href="https://drive.google.com/uc?export=download&id=1TTl4gvrTvNbegnml4CoGoKOOd6O8-PGs"> SVG Figures</a>
-            /
-            <a href="https://docs.google.com/presentation/d/11zhB6PI-Dp6Ogmr4IcI6fbvbqNqLyYcz/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
-                Figures</a>
-        <li> Chapter 8 - Measuring performance: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap8PDF.zip">PDF Figures</a> / <a
-                href="https://drive.google.com/uc?export=download&id=19eQOnygd_l0DzgtJxXuYnWa4z7QKJrJx"> SVG Figures</a>
-            /
-            <a href="https://docs.google.com/presentation/d/1SHRmJscDLUuQrG7tmysnScb3ZUAqVMZo/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
-                Figures</a>
-        <li> Chapter 9 - Regularization: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap9PDF.zip">PDF
-            Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1LprgnUGL7xAM9-jlGZC9LhMPeefjY0r0">
-            SVG
-            Figures</a> / <a
-                href="https://docs.google.com/presentation/d/1VwIfvjpdfTny6sEfu4ZETwCnw6m8Eg-5/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
-            Figures</a>
-        <li> Chapter 10 - Convolutional networks: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap10PDF.zip">PDF Figures</a> / <a
-                href="https://drive.google.com/uc?export=download&id=1-Wb3VzaSvVeRzoUzJbI2JjZE0uwqupM9"> SVG Figures</a>
-            /
-            <a href="https://docs.google.com/presentation/d/1MtfKBC4Y9hWwGqeP6DVwUNbi1j5ncQCg/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
-                Figures</a>
-        <li> Chapter 11 - Residual networks: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap11PDF.zip">PDF Figures</a> / <a
-                href="https://drive.google.com/uc?export=download&id=1Mr58jzEVseUAfNYbGWCQyDtEDwvfHRi1"> SVG Figures</a>
-            /
-            <a href="https://docs.google.com/presentation/d/1saY8Faz0KTKAAifUrbkQdLA2qkyEjOPI/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
-                Figures</a>
-        <li> Chapter 12 - Transformers: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap12PDF.zip">PDF
-            Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1txzOVNf8-jH4UfJ6SLnrtOfPd1Q3ebzd">
-            SVG
-            Figures</a> / <a
-                href="https://docs.google.com/presentation/d/1GVNvYWa0WJA6oKg89qZre-UZEhABfm0l/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
-            Figures</a>
-        <li> Chapter 13 - Graph neural networks: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap13PDF.zip">PDF Figures</a> / <a
-                href="https://drive.google.com/uc?export=download&id=1lQIV6nRp6LVfaMgpGFhuwEXG-lTEaAwe"> SVG Figures</a>
-            /
-            <a href="https://docs.google.com/presentation/d/1YwF3U82c1mQ74c1WqHVTzLZ0j7GgKaWP/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
-                Figures</a>
-        <li> Chapter 14 - Unsupervised learning: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap14PDF.zip">PDF Figures</a> / <a
-                href="https://drive.google.com/uc?export=download&id=1aMbI6iCuUvOywqk5pBOmppJu1L1anqsM"> SVG Figures</a>
-            /
-            <a href="https://docs.google.com/presentation/d/1A-lBGv3NHl4L32NvfFgy1EKeSwY-0UeB/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">
-                PowerPoint Figures</a>
-        <li> Chapter 15 - Generative adversarial networks: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap15PDF.zip">PDF Figures</a> / <a
-                href="https://drive.google.com/uc?export=download&id=1EErnlZCOlXc3HK7m83T2Jh_0NzIUHvtL"> SVG Figures</a>
-            /
-            <a href="https://docs.google.com/presentation/d/10Ernk41ShOTf4IYkMD-l4dJfKATkXH4w/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
-                Figures</a>
-        <li> Chapter 16 - Normalizing flows: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap16PDF.zip">PDF Figures</a> / <a
-                href="https://drive.google.com/uc?export=download&id=1B9bxtmdugwtg-b7Y4AdQKAIEVWxjx8l3"> SVG Figures</a>                                                                                                     
-            /
-            <a href="https://docs.google.com/presentation/d/1nLLzqb9pdfF_h6i1HUDSyp7kSMIkSUUA/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
-                Figures</a>
-        <li> Chapter 17 - Variational autoencoders: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap17PDF.zip">PDF Figures</a> / <a
-                href="https://drive.google.com/uc?export=download&id=1SNtNIY7khlHQYMtaOH-FosSH3kWwL4b7"> SVG Figures</a>
-            /
-            <a href="https://docs.google.com/presentation/d/1lQE4Bu7-LgvV2VlJOt_4dQT-kusYl7Vo/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
-                Figures</a>
-        <li> Chapter 18 - Diffusion models: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap18PDF.zip">PDF Figures</a> / <a
-                href="https://drive.google.com/uc?export=download&id=1A-pIGl4PxjVMYOKAUG3aT4a8wD3G-q_r"> SVG Figures</a>
-            /
-            <a href="https://docs.google.com/presentation/d/1x_ufIBtVPzWUvRieKMkpw5SdRjXWwdfR/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">
-            PowerPoint Figures</a>
-        <li> Chapter 19 - Deep reinforcement learning: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap19PDF.zip">PDF Figures</a> / <a
-                href="https://drive.google.com/uc?export=download&id=1a5WUoF7jeSgwC_PVdckJi1Gny46fCqh0"> SVG Figures</a>
-            /
-            <a href="https://docs.google.com/presentation/d/1TnYmVbFNhmMFetbjyfXGmkxp1EHauMqr/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">
-                PowerPoint Figures </a>
-        <li> Chapter 20 - Why does deep learning work?: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap20PDF.zip">PDF Figures</a> / <a
-                href="https://drive.google.com/uc?export=download&id=1M2d0DHEgddAQoIedKSDTTt7m1ZdmBLQ3"> SVG Figures</a>
-            /
-            <a href="https://docs.google.com/presentation/d/1coxF4IsrCzDTLrNjRagHvqB_FBy10miA/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">
-                PowerPoint Figures</a>
-        <li> Chapter 21 - Deep learning and ethics: <a
-                href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap21PDF.zip">PDF Figures</a> / <a
-                href="https://drive.google.com/uc?export=download&id=1jixmFfwmZkW_UVYzcxmDcMsdFFtnZ0bU"> SVG Figures</a>/
-            <a
-                    href="https://docs.google.com/presentation/d/1EtfzanZYILvi9_-Idm28zD94I_6OrN9R/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
-                Figures</a>
-        <li> Appendices - <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLAppendixPDF.zip">PDF
-            Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1k2j7hMN40ISPSg9skFYWFL3oZT7r8v-l">
-            SVG
-            Figures</a> / <a
-                href="https://docs.google.com/presentation/d/1_2cJHRnsoQQHst0rwZssv-XH4o5SEHks/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">Powerpoint
-            Figures</a>
-    </ul>
-
-    Instructions for editing figures / equations can be found <a
-        href="https://drive.google.com/file/d/1T_MXXVR4AfyMnlEFI-UVDh--FXI5deAp/view?usp=sharing">here</a>.
-
-    <p> My slides for 20 lecture undergraduate deep learning course:</p>
-    <ul>
-        <li><a href="https://drive.google.com/uc?export=download&id=17RHb11BrydOvxSFNbRIomE1QKLVI087m">1. Introduction</a></li>
-        <li><a href="https://drive.google.com/uc?export=download&id=1491zkHULC7gDfqlV6cqUxyVYXZ-de-Ub">2. Supervised Learning</a></li>
-        <li><a href="https://drive.google.com/uc?export=download&id=1XkP1c9EhOBowla1rT1nnsDGMf2rZvrt7">3. Shallow Neural Networks</a></li>
-        <li><a href="https://drive.google.com/uc?export=download&id=1e2ejfZbbfMKLBv0v-tvBWBdI8gO3SSS1">4. Deep Neural Networks</a></li>
-        <li><a href="https://drive.google.com/uc?export=download&id=1fxQ_a1Q3eFPZ4kPqKbak6_emJK-JfnRH">5. Loss Functions</a></li>
-        <li><a href="https://drive.google.com/uc?export=download&id=17QQ5ZzXBtR_uCNCUU1gPRWWRUeZN9exW">6. Fitting Models</a></li>
-        <li><a href="https://drive.google.com/uc?export=download&id=1hC8JUCOaFWiw3KGn0rm7nW6mEq242QDK">7. Computing Gradients</a></li>
-        <li><a href="https://drive.google.com/uc?export=download&id=1tSjCeAVg0JCeBcPgDJDbi7Gg43Qkh9_d">7b. Initialization</a></li>
-        <li><a href="https://drive.google.com/uc?export=download&id=1RVZW3KjEs0vNSGx3B2fdizddlr6I0wLl">8. Performance</a></li>
-        <li><a href="https://drive.google.com/uc?export=download&id=1LTicIKPRPbZRkkg6qOr1DSuOB72axood">9. Regularization</a></li>
-        <li><a href="https://drive.google.com/uc?export=download&id=1bGVuwAwrofzZdfvj267elIzkYMIvYFj0">10. Convolutional Networks</a></li>
-        <li><a href="https://drive.google.com/uc?export=download&id=14w31QqWRDix1GdUE-na0_E0kGKBhtKzs">11. Image Generation</a></li>
-        <li><a href="https://drive.google.com/uc?export=download&id=1af6bTTjAbhDYfrDhboW7Fuv52Gk9ygKr">12. Transformers and LLMs</a></li>
-    </ul>
-
-    <h2>Resources for students</h2>
-
-    <p>Answers to selected questions: <a
-            href="https://github.com/udlbook/udlbook/raw/main/UDL_Answer_Booklet_Students.pdf">PDF</a>
-    </p>
-    <p>Python notebooks: (Early ones more thoroughly tested than later ones!)</p>
-
-    <ul>
-        <li> Notebook 1.1 - Background mathematics: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap01/1_1_BackgroundMathematics.ipynb">ipynb/colab</a>
-        </li>
-        <li> Notebook 2.1 - Supervised learning: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap02/2_1_Supervised_Learning.ipynb">ipynb/colab</a>
-        </li>
-        <li> Notebook 3.1 - Shallow networks I: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_1_Shallow_Networks_I.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 3.2 - Shallow networks II: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_2_Shallow_Networks_II.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 3.3 - Shallow network regions: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_3_Shallow_Network_Regions.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 3.4 - Activation functions: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_4_Activation_Functions.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 4.1 - Composing networks: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap04/4_1_Composing_Networks.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 4.2 - Clipping functions: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap04/4_2_Clipping_functions.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 4.3 - Deep networks: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap04/4_3_Deep_Networks.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 5.1 - Least squares loss: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap05/5_1_Least_Squares_Loss.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 5.2 - Binary cross-entropy loss: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap05/5_2_Binary_Cross_Entropy_Loss.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 5.3 - Multiclass cross-entropy loss: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap05/5_3_Multiclass_Cross_entropy_Loss.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 6.1 - Line search: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_1_Line_Search.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 6.2 - Gradient descent: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_2_Gradient_Descent.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 6.3 - Stochastic gradient descent: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_3_Stochastic_Gradient_Descent.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 6.4 - Momentum: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_4_Momentum.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 6.5 - Adam: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_5_Adam.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 7.1 - Backpropagation in toy model: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap07/7_1_Backpropagation_in_Toy_Model.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 7.2 - Backpropagation: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap07/7_2_Backpropagation.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 7.3 - Initialization: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap07/7_3_Initialization.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 8.1 - MNIST-1D performance: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap08/8_1_MNIST_1D_Performance.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 8.2 - Bias-variance trade-off: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap08/8_2_Bias_Variance_Trade_Off.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 8.3 - Double descent: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap08/8_3_Double_Descent.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 8.4 - High-dimensional spaces: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap08/8_4_High_Dimensional_Spaces.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 9.1 - L2 regularization: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap09/9_1_L2_Regularization.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 9.2 - Implicit regularization: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap09/9_2_Implicit_Regularization.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 9.3 - Ensembling: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap09/9_3_Ensembling.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 9.4 - Bayesian approach: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap09/9_4_Bayesian_Approach.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 9.5 - Augmentation <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap09/9_5_Augmentation.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 10.1 - 1D convolution: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap10/10_1_1D_Convolution.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 10.2 - Convolution for MNIST-1D: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap10/10_2_Convolution_for_MNIST_1D.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 10.3 - 2D convolution: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap10/10_3_2D_Convolution.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 10.4 - Downsampling & upsampling: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap10/10_4_Downsampling_and_Upsampling.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 10.5 - Convolution for MNIST: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap10/10_5_Convolution_For_MNIST.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 11.1 - Shattered gradients: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap11/11_1_Shattered_Gradients.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 11.2 - Residual networks: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap11/11_2_Residual_Networks.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 11.3 - Batch normalization: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap11/11_3_Batch_Normalization.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 12.1 - Self-attention: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap12/12_1_Self_Attention.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 12.2 - Multi-head self-attention: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap12/12_2_Multihead_Self_Attention.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 12.3 - Tokenization: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap12/12_3_Tokenization.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 12.4 - Decoding strategies: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap12/12_4_Decoding_Strategies.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 13.1 - Encoding graphs: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap13/13_1_Graph_Representation.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 13.2 - Graph classification : <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap13/13_2_Graph_Classification.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 13.3 - Neighborhood sampling: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap13/13_3_Neighborhood_Sampling.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 13.4 - Graph attention: <a
-                href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap13/13_4_Graph_Attention_Networks.ipynb">ipynb/colab </a>
-        </li>
-        <li> Notebook 15.1 - GAN toy example: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap15/15_1_GAN_Toy_Example.ipynb">ipynb/colab </a></li>
-        <li> Notebook 15.2 - Wasserstein distance: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap15/15_2_Wasserstein_Distance.ipynb">ipynb/colab </a></li>
-        <li> Notebook 16.1 - 1D normalizing flows: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap16/16_1_1D_Normalizing_Flows.ipynb">ipynb/colab </a></li>
-        <li> Notebook 16.2 - Autoregressive flows: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap16/16_2_Autoregressive_Flows.ipynb">ipynb/colab </a></li>
-        <li> Notebook 16.3 - Contraction mappings: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap16/16_3_Contraction_Mappings.ipynb">ipynb/colab </a></li>
-        <li> Notebook 17.1 - Latent variable models: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap17/17_1_Latent_Variable_Models.ipynb">ipynb/colab </a></li>
-        <li> Notebook 17.2 - Reparameterization trick: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap17/17_2_Reparameterization_Trick.ipynb">ipynb/colab </a></li>
-        <li> Notebook 17.3 - Importance sampling: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap17/17_3_Importance_Sampling.ipynb">ipynb/colab </a></li>
-        <li> Notebook 18.1 - Diffusion encoder: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap18/18_1_Diffusion_Encoder.ipynb">ipynb/colab </a></li>
-        <li> Notebook 18.2 - 1D diffusion model: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap18/18_2_1D_Diffusion_Model.ipynb">ipynb/colab </a></li>
-        <li> Notebook 18.3 - Reparameterized model: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap18/18_3_Reparameterized_Model.ipynb">ipynb/colab </a></li>
-        <li> Notebook 18.4 - Families of diffusion models: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap18/18_4_Families_of_Diffusion_Models.ipynb">ipynb/colab </a></li>
-        <li> Notebook 19.1 - Markov decision processes: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap19/19_1_Markov_Decision_Processes.ipynb">ipynb/colab </a></li>
-        <li> Notebook 19.2 - Dynamic programming: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap19/19_2_Dynamic_Programming.ipynb">ipynb/colab </a></li>
-        <li> Notebook 19.3 - Monte-Carlo methods: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap19/19_3_Monte_Carlo_Methods.ipynb">ipynb/colab </a></li>
-        <li> Notebook 19.4 - Temporal difference methods: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap19/19_4_Temporal_Difference_Methods.ipynb">ipynb/colab </a></li>
-        <li> Notebook 19.5 - Control variates: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap19/19_5_Control_Variates.ipynb">ipynb/colab </a></li>
-        <li> Notebook 20.1 - Random data: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap20/20_1_Random_Data.ipynb">ipynb/colab </a></li>
-        <li> Notebook 20.2 - Full-batch gradient descent: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap20/20_2_Full_Batch_Gradient_Descent.ipynb">ipynb/colab </a></li>
-        <li> Notebook 20.3 - Lottery tickets: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap20/20_3_Lottery_Tickets.ipynb">ipynb/colab </a></li>
-        <li> Notebook 20.4 - Adversarial attacks: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap20/20_4_Adversarial_Attacks.ipynb">ipynb/colab </a></li>
-        <li> Notebook 21.1 - Bias mitigation: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap21/21_1_Bias_Mitigation.ipynb">ipynb/colab </a></li>
-        <li> Notebook 21.2 - Explainability: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap21/21_2_Explainability.ipynb">ipynb/colab </a></li>
-    </ul>
-
-
-    <br>
-    <h2>Citation</h2>
-    <pre><code>
- @book{prince2023understanding,
- author = "Simon J.D. Prince",
- title = "Understanding Deep Learning",
- publisher = "MIT Press",
- year = 2023,
- url = "http://udlbook.com"
-}
-    </code></pre>
-</div>
-</body>

package.json

@@ -0,0 +1,50 @@
+{
+  "name": "react-website-smooth-scroll",
+  "version": "0.1.0",
+  "private": true,
+  "homepage": "https://udlbook.github.io/udlbook",
+  "dependencies": {
+    "@fortawesome/fontawesome-svg-core": "^6.5.1",
+    "@testing-library/jest-dom": "^5.15.1",
+    "@testing-library/react": "^11.2.7",
+    "@testing-library/user-event": "^12.8.3",
+    "react": "^17.0.2",
+    "react-dom": "^17.0.2",
+    "react-icons": "^5.0.1",
+    "react-router-dom": "^6.0.2",
+    "react-scripts": "4.0.3",
+    "react-scroll": "^1.8.4",
+    "styled-components": "^5.3.3",
+    "url-loader": "^4.1.1",
+    "web-vitals": "^1.1.2"
+  },
+  "scripts": {
+    "start": "react-scripts --openssl-legacy-provider start",
+    "build": "react-scripts --openssl-legacy-provider build",
+    "test": "react-scripts test",
+    "eject": "react-scripts eject",
+    "predeploy": "npm run build",
+    "deploy": "gh-pages -d build"
+  },
+  "eslintConfig": {
+    "extends": [
+      "react-app",
+      "react-app/jest"
+    ]
+  },
+  "browserslist": {
+    "production": [
+      ">0.2%",
+      "not dead",
+      "not op_mini all"
+    ],
+    "development": [
+      "last 1 chrome version",
+      "last 1 firefox version",
+      "last 1 safari version"
+    ]
+  },
+  "devDependencies": {
+    "gh-pages": "^6.1.1"
+  }
+}

public/index.html

@@ -0,0 +1,46 @@
+<!DOCTYPE html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8" />
+    <link rel="icon" href="%PUBLIC_URL%/favicon.ico" />
+    <meta name="viewport" content="width=device-width, initial-scale=1" />
+    <meta name="theme-color" content="#000000" />
+    <meta
+      name="description"
+      content="Web site created using create-react-app"
+    />
+    <link rel="apple-touch-icon" href="%PUBLIC_URL%/logo192.png" />
+    <!--
+      manifest.json provides metadata used when your web app is installed on a
+      user's mobile device or desktop. See https://developers.google.com/web/fundamentals/web-app-manifest/
+    -->
+    <link rel="manifest" href="%PUBLIC_URL%/manifest.json" />
+    <!--
+      Notice the use of %PUBLIC_URL% in the tags above.
+      It will be replaced with the URL of the `public` folder during the build.
+      Only files inside the `public` folder can be referenced from the HTML.
+
+      Unlike "/favicon.ico" or "favicon.ico", "%PUBLIC_URL%/favicon.ico" will
+      work correctly both with client-side routing and a non-root public URL.
+      Learn how to configure a non-root public URL by running `npm run build`.
+    -->
+    <link rel="preconnect" href="https://fonts.googleapis.com">
+<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
+<link href="https://fonts.googleapis.com/css2?family=Encode+Sans+Expanded:wght@400;700&display=swap" rel="stylesheet">
+    <title>Understanding Deep Learning</title>
+  </head>
+  <body>
+    <noscript>You need to enable JavaScript to run this app.</noscript>
+    <div id="root"></div>
+    <!--
+      This HTML file is a template.
+      If you open it directly in the browser, you will see an empty page.
+
+      You can add webfonts, meta tags, or analytics to this file.
+      The build step will place the bundled scripts into the <body> tag.
+
+      To begin the development, run `npm start` or `yarn start`.
+      To create a production bundle, use `npm run build` or `yarn build`.
+    -->
+  </body>
+</html>

public/manifest.json

@@ -0,0 +1,25 @@
+{
+  "short_name": "React App",
+  "name": "Create React App Sample",
+  "icons": [
+    {
+      "src": "favicon.ico",
+      "sizes": "64x64 32x32 24x24 16x16",
+      "type": "image/x-icon"
+    },
+    {
+      "src": "logo192.png",
+      "type": "image/png",
+      "sizes": "192x192"
+    },
+    {
+      "src": "logo512.png",
+      "type": "image/png",
+      "sizes": "512x512"
+    }
+  ],
+  "start_url": ".",
+  "display": "standalone",
+  "theme_color": "#000000",
+  "background_color": "#ffffff"
+}

public/robots.txt

@@ -0,0 +1,3 @@
+# https://www.robotstxt.org/robotstxt.html
+User-agent: *
+Disallow:

src/App.css

@@ -0,0 +1,6 @@
+*{
+    box-sizing: border-box;
+    margin: 0;
+    padding: 0 ;
+    font-family: 'Encode Sans Expanded', sans-serif;
+}

src/App.js

@@ -0,0 +1,19 @@
+import './App.css';
+import {BrowserRouter as Router, Routes, Route} from 'react-router-dom'
+import Home from './pages';
+
+
+
+
+function App() {
+  return (
+    <Router>
+      <Routes>
+        <Route exact path="/udlbook/" element ={<Home/>} />      
+      </Routes>
+      
+    </Router>
+  );
+}
+
+export default App;

src/components/ButtonElement.js

@@ -0,0 +1,23 @@
+import styled from  'styled-components'
+import {Link} from 'react-scroll'
+
+
+export const Button= styled(Link)` 
+    border-radius: 50px;
+    background: ${({primary}) => (primary ? '#01BF71' : '#010606')};
+    white-space: nowrap;
+    padding: ${({big}) => (big? ' 14px 48px': '12px 30px')};
+    color: ${({dark}) => (dark ? '#010106': '#fff')};
+    font-size: $${({fontBig}) => (fontBig ? '20px' : '16px')};
+    outline: none;
+    border: none;
+    cursor: pointer;
+    display: flex;
+    justify-content: center;
+    align-items: center;
+    transition: all 0.2s ease-in-out;
+    &:hover {
+        transition: all 0.2s ease-in-out;
+        background: ${({primary}) => (primary ? '#fff' : '#01BF71')}
+    }
+`

src/components/Footer/FooterElements.js

@@ -0,0 +1,142 @@
+import styled from 'styled-components'
+import {Link} from 'react-router-dom'
+
+export const FooterContainer = styled.footer` 
+    background-color: #101522;
+`
+
+export const FooterWrap = styled.div` 
+    padding: 48x 24px;
+    display: flex;
+    flex-direction: column;
+    justify-content: center;
+    align-items: center;
+    max-width: 1100px;
+    margin: 0 auto;
+`
+
+export const FooterLinksContainer = styled.div` 
+    display: flex;
+    justify-content: center;
+
+    @media screen and (max-width: 820px){
+        padding-top: 32px;
+    }
+`
+
+export const FooterLinksWrapper = styled.div` 
+    display: flex;
+    @media screen and (max-width: 820px){
+        flex-direction: column;
+    }
+`
+
+export const FooterLinkItems = styled.div` 
+    display: flex;
+    flex-direction: column;
+    align-items: flex-start;
+    margin: 16px;
+    text-align: left;
+    width: 160px;
+    box-sizing: border-box;
+    color: #fff;
+
+    @media screen and (max-width: 420px){
+        margin: 0;
+        padding: 10px;
+        width: 100%;
+    }
+`
+
+export const FooterLinkTitle = styled.h1` 
+    font-size: 14px;
+    margin-bottom: 16px ; 
+`
+
+export const FooterLink = styled(Link)`
+    color: #ffffff;
+    text-decoration: none;
+    margin-bottom: 0.5rem;
+    font-size: 14px;
+
+    &:hover{
+        color: #01bf71;
+        transition: 0.3s ease-in-out;
+    }
+`
+
+export const SocialMedia = styled.section` 
+    max-width: 1000px;
+    width: 100%;
+`
+
+export const SocialMediaWrap = styled.div` 
+    display: flex;
+    justify-content: space-between;
+    align-items: center;
+    max-width: 1100px; 
+    margin: 20px auto 0 auto ; 
+
+    @media screen and (max-width: 820px){
+        flex-direction: column;
+    }
+`
+
+export const SocialAttrWrap = styled.div` 
+    color: #fff;
+    display: flex;
+    justify-content: center;
+    align-items: center;
+    max-width: 1100px; 
+    margin: 10px auto 0 auto ; 
+
+    @media screen and (max-width: 820px){
+        flex-direction: column;
+    }
+`
+
+export const SocialLogo = styled(Link)` 
+    color: #fff;
+    justify-self: start;
+    cursor: pointer;
+    text-decoration: none;
+    font-size: 1.5rem;
+    display: flex;
+    align-items: center;
+    margin-bottom: 16px;
+    font-weight: bold;
+    @media screen and (max-width: 768px) {
+        font-size: 20px;
+    }
+`
+
+export const WebsiteRights = styled.small` 
+    color: #fff ; 
+    margin-bottom: 8px ; 
+`
+
+export const SocialIcons = styled.div`
+    display: flex;
+    justify-content: space-between;
+    align-items: center;
+    width: 60px;
+    margin-bottom: 8px ; 
+`
+
+export const SocialIconLink = styled.a` 
+    color: #fff;
+    font-size: 24px;
+`
+
+export const FooterImgWrap = styled.div` 
+    max-width: 555px;
+    height: 100%;
+`
+
+export const FooterImg = styled.img`
+  width: 100%;
+  margin-top: 0;
+  margin-right: 0;
+  margin-left: 10px;
+  padding-right: 0;
+`;

src/components/Footer/index.js

@@ -0,0 +1,42 @@
+import React from 'react'
+import { FaLinkedin} from 'react-icons/fa'
+import { FooterContainer, FooterWrap, FooterImg } from './FooterElements'
+import { SocialMedia,  SocialMediaWrap, SocialIcons, SocialIconLink, WebsiteRights, SocialLogo } from './FooterElements'
+import { animateScroll as scroll } from 'react-scroll'
+import twitterImg from '../../images/square-x-twitter.svg'
+
+const Footer = () => {
+    const toggleHome = () => {
+        scroll.scrollToTop();
+    }
+
+    return (
+        <>
+            <FooterContainer>
+                <FooterWrap>
+                    <SocialMedia>
+                        <SocialMediaWrap>
+                            <SocialLogo to='/udlbook/' onClick={toggleHome}>
+                                Understanding Deep Learning
+                            </SocialLogo>
+                            <WebsiteRights>©{new Date().getFullYear()} Simon J.D. Prince</WebsiteRights>
+                            <WebsiteRights>
+                            Images by StorySet on FreePik: <a href="https://www.freepik.com/free-vector/hand-coding-concept-illustration_21864184.htm#query=coding&position=17&from_view=search&track=sph&uuid=5896d847-38e4-4cb9-8fe1-103041c7c933"> [1] </a> <a href="https://www.freepik.com/free-vector/mathematics-concept-illustration_10733824.htm#query=professor&position=13&from_view=search&track=sph&uuid=5b1a188a-64c5-45af-aae2-8573bc1bed3c">[2]</a> <a href="https://www.freepik.com/free-vector/content-concept-illustration_7171429.htm#query=media&position=3&from_view=search&track=sph&uuid=c7e35cf2-d85d-4bba-91a6-1cd883dcf153"> [3]</a>  <a href="https://www.freepik.com/free-vector/library-concept-illustration_9148008.htm#query=library&position=40&from_view=search&track=sph&uuid=abecc792-b6b2-4ec0-b318-5e6cc73ba649"> [4]</a> 
+                            </WebsiteRights>
+                            <SocialIcons>
+                                <SocialIconLink href="https://twitter.com/SimonPrinceAI" target="_blank" aria-label="Twitter">
+                                    <FooterImg src={twitterImg} alt="twitter"/>
+                                </SocialIconLink>
+                                <SocialIconLink href="https://www.linkedin.com/in/simon-prince-615bb9165/" target="_blank" aria-label="LinkedIn">
+                                    <FaLinkedin/>
+                                </SocialIconLink>
+                            </SocialIcons>
+                        </SocialMediaWrap>
+                    </SocialMedia>
+                </FooterWrap>  
+            </FooterContainer>  
+        </>
+    )
+}
+
+export default Footer

src/components/HeroSection/HeroElements.js

@@ -0,0 +1,304 @@
+import styled from "styled-components";
+
+export const HeroContainer = styled.div`
+    background: #57c6d1;
+    display: flex;
+    justify-content: center;
+    align-items: center;
+    padding: 0 0px;
+    position: static;
+    z-index: 1;
+    }
+`
+
+
+
+export const HeroContent = styled.div` 
+    z-index: 3; 
+    width: 100% ; 
+    max-width: 1100px;
+    position: static;
+    padding: 8px 24px;
+    margin: 80px 0px;
+    display: flex;
+    flex-direction: column;
+    align-items: center ; 
+`
+export const HeroH1 = styled.h1` 
+    color: #fff;
+    font-size: 48px;
+    text-align: center;
+
+    @media screen and (max-width: 768px) {
+        font-size: 40px;
+    }
+
+    @media screen and (max-width: 480px) {
+        font-size: 32px;
+    }
+
+`
+
+export const HeroP = styled.p` 
+    margin-top: 24px;
+    color: #fff;
+    font-size: 24px ; 
+    text-align: center ; 
+    max-width: 600px ; 
+
+    
+    @media screen and (max-width: 768px) {
+        font-size: 24px;
+    }
+
+    @media screen and (max-width: 480px) {
+        font-size: 18px;
+    }
+`
+
+export const HeroBtnWrapper = styled.div`
+    margin-top: 32px ; 
+    display: flex;
+    flex-direction: column ; 
+    align-items: center ; 
+`
+
+
+export const HeroRow = styled.div` 
+    display: grid; 
+    grid-auto-columns: minmax(auto, 1fr);
+    align-items: top;
+    grid-template-areas:  'col1 col2' };
+
+    @media screen and (max-width: 768px){
+        grid-template-areas: 'col2' 'col1';
+    }
+`
+
+
+export const HeroNewsItem = styled.div`
+    margin-left: 4px;
+    color: #000000;
+    font-size: 16px;
+    // line-height: 16px;
+    margin-bottom: 16px;
+    display: flex;  
+    justify-content: start;
+
+`
+export const HeroNewsItemDate = styled.div`
+    width: 20%;
+    margin-right: 20px ; 
+    @media screen and (max-width: 768px) {
+        font-size: 12px;
+    }
+    @media screen and (max-width: 480px) {
+        font-size: 12px;
+    }
+`
+
+export const HeroNewsItemContent = styled.div`
+    width: 80%; 
+    color: #000000;
+
+    @media screen and (max-width: 768px) {
+        font-size: 12px;
+    }
+
+    @media screen and (max-width: 480px) {
+        font-size: 12px;
+    }
+`
+
+
+
+export const HeroColumn1 = styled.div` 
+    margin-bottom: 15px;
+    margin-left: 12px;
+    margin-top: 60px;
+    padding: 10px 15px;
+    padding: 0 15px;
+    grid-area: col1;
+    align-items:left;
+    display: flex;
+    flex-direction:column;
+    justify-content: space-between;
+`
+
+
+export const HeroColumn2 = styled.div` 
+    margin-bottom: 15px;
+    padding: 0 15px;
+    grid-area: col2;
+    display: flex;
+    align-items:center;
+    flex-direction:column;
+`
+
+export const TextWrapper = styled.div`
+    max-width: 540px ; 
+    padding-top: 0;
+    padding-bottom: 0;
+`
+export const HeroImgWrap = styled.div` 
+    max-width: 555px;
+    height: 100%;
+`
+
+export const Img = styled.img`
+  width: 100%;
+  margin-top: 0;
+  margin-right: 0;
+  margin-left: 10px;
+  padding-right: 0;
+`;
+
+export const HeroDownloadsImg = styled.img`
+    margin-top: 5px;
+    margin-right: 0;
+    margin-left: 0;
+    padding-right: 0;
+    margin-bottom: 10px;
+`
+
+export const HeroLink = styled.a`
+  color: #fff;
+  text-decoration: none;
+  padding: 0.6rem 0rem 0rem 0rem;
+  cursor: pointer;
+  position:relative ; 
+
+  &:before{
+    position: absolute;
+    margin: 0 auto;
+    top: 100%;
+    left: 0;
+    width: 100%;
+    height: 2px;
+    background-color: #fff;
+    content: '';
+    opacity: .3;
+    -webkit-transform: scaleX(1);
+    transition-property: opacity, -webkit-transform;
+    transition-duration: .3s;
+}
+
+    &:hover:before {
+    opacity: 1;
+    -webkit-transform: scaleX(1.05);
+  }
+`;
+
+//   color: #fff;
+//   text-decoration: none;
+//   padding: 0.1rem 0rem;
+//   height: 100%;
+//   cursor: pointer;
+//   position:relative ; 
+
+//   &:before{
+//     position: absolute;
+//     margin: 0 auto;
+//     top: 100%;
+//     left: 0;
+//     width: 100%;
+//     height: 2px;
+//     background-color: #000;
+//     content: '';
+//     opacity: .3;
+//     -webkit-transform: scaleX(1);
+//     transition-property: opacity, -webkit-transform;
+//     transition-duration: .3s;
+// }
+
+//     &:hover:before {
+//     opacity: 1;
+//     -webkit-transform: scaleX(1.05);
+//   }
+// `;
+
+export const UDLLink = styled.a`
+    text-decoration: none;
+    color: #000;
+    font-weight: 300;
+    margin: 0 2px;
+    position: relative;
+
+    &:before{
+        position: absolute;
+        margin: 0 auto;
+        top: 100%;
+        left: 0;
+        width: 100%;
+        height: 2px;
+        background-color: #000;
+        content: '';
+        opacity: .3;
+        -webkit-transform: scaleX(1);
+        transition-property: opacity, -webkit-transform;
+        transition-duration: .3s;
+    }
+
+    &:hover:before {
+        opacity: 1;
+        -webkit-transform: scaleX(1.05);
+      }
+`
+
+
+
+
+export const HeroNewsTitle = styled.div` 
+    margin-left: 0px;
+    color: #000000;
+    font-size: 16px;
+    font-weight: bold;
+    line-height: 16px;
+    margin-bottom: 36px; 
+
+    @media screen and (max-width: 768px) {
+        font-size: 24px;
+    }
+
+    @media screen and (max-width: 480px) {
+        font-size: 18px;
+    }
+`
+
+export const HeroCitationTitle = styled.div` 
+    margin-left: 0px;
+    color: #000000;
+    font-size: 16px;
+    font-weight: bold;
+    line-height: 16px;
+    margin-bottom: 10px; 
+    margin-top:36px;
+
+    @media screen and (max-width: 768px) {
+        font-size: 24px;
+    }
+
+    @media screen and (max-width: 480px) {
+        font-size: 18px;
+    }
+`
+
+
+export const HeroNewsBlock = styled.div`
+
+`
+export const HeroCitationBlock = styled.div`
+    font-size: 14px;
+    margin-bottom: 0px;
+    margin-top: 0px;
+
+`
+
+
+
+
+export const HeroFollowBlock = styled.div`
+@media screen and (max-width: 768px) {
+    font-size: 14px;
+}
+`

src/components/HeroSection/index.js

@@ -0,0 +1,94 @@
+import React from 'react'
+import { HeroContainer, HeroNewsBlock, HeroCitationBlock, HeroCitationTitle, HeroFollowBlock, HeroDownloadsImg, HeroLink, HeroRow, HeroColumn1, HeroColumn2, HeroContent, Img, HeroImgWrap, HeroNewsTitle, HeroNewsItem, HeroNewsItemDate, HeroNewsItemContent, UDLLink} from './HeroElements'
+import img from '../../images/F23.prince.learning.turquoise.jpg'
+
+const HeroSection = () => {
+
+
+    const citation = `
+@book{prince2023understanding,
+    author = "Simon J.D. Prince",
+    title = "Understanding Deep Learning",
+    publisher = "The MIT Press",
+    year = 2023,
+    url = "http://udlbook.com"}
+    `
+ 
+    return (
+        <HeroContainer id="home">
+            <HeroContent>
+                <HeroRow>
+                    <HeroColumn1>
+                        <HeroNewsBlock>
+                            <HeroNewsTitle>RECENT NEWS:</HeroNewsTitle>
+
+                            <HeroNewsItem>
+                                <HeroNewsItemDate>03/12/24</HeroNewsItemDate>
+                                <HeroNewsItemContent> Book now available again.</HeroNewsItemContent>
+                            </HeroNewsItem>
+
+                            <HeroNewsItem>
+                                <HeroNewsItemDate>02/21/24</HeroNewsItemDate>
+                                <HeroNewsItemContent>New blog about the <UDLLink href="https://www.borealisai.com/research-blogs/the-neural-tangent-kernel/">Neural Tangent Kernel.</UDLLink></HeroNewsItemContent>
+                            </HeroNewsItem>
+                            <HeroNewsItem>
+                                <HeroNewsItemDate>02/15/24</HeroNewsItemDate>
+                                <HeroNewsItemContent> First printing of book has sold out in most places. Second printing available mid-March.</HeroNewsItemContent>
+                            </HeroNewsItem>
+
+
+                            <HeroNewsItem>
+                                <HeroNewsItemDate>01/29/24</HeroNewsItemDate>
+                                <HeroNewsItemContent> New blog about <UDLLink href="https://www.borealisai.com/research-blogs/gradient-flow/"> gradient flow </UDLLink> published.</HeroNewsItemContent>
+                            </HeroNewsItem>
+
+                            <HeroNewsItem>
+                                <HeroNewsItemDate>12/26/23</HeroNewsItemDate>
+                                <HeroNewsItemContent> Machine Learning Street Talk  <UDLLink href="https://www.youtube.com/watch?v=sJXn4Cl4oww"> podcast </UDLLink>  discussing book.</HeroNewsItemContent>
+                            </HeroNewsItem>
+
+                            <HeroNewsItem>
+                                <HeroNewsItemDate>12/19/23</HeroNewsItemDate>
+                                <HeroNewsItemContent>Deeper Insights <UDLLink href="https://podcasts.apple.com/us/podcast/understanding-deep-learning-with-simon-prince/id1669436318?i=1000638269385">podcast</UDLLink> discussing book.</HeroNewsItemContent>
+                            </HeroNewsItem>
+
+                            <HeroNewsItem>
+                                <HeroNewsItemDate>12/06/23</HeroNewsItemDate>
+                                <HeroNewsItemContent> I did an <UDLLink href="https://www.borealisai.com/news/understanding-deep-learning/">interview</UDLLink> discussing the book with Borealis AI.</HeroNewsItemContent>
+                            </HeroNewsItem>
+
+                            <HeroNewsItem>
+                                <HeroNewsItemDate>12/05/23</HeroNewsItemDate>
+                                <HeroNewsItemContent> Book released by <UDLLink href="https://mitpress.mit.edu/9780262048644/understanding-deep-learning/">The MIT Press</UDLLink>.</HeroNewsItemContent>
+                            </HeroNewsItem>
+                        </HeroNewsBlock>
+                        <HeroCitationTitle>CITATION:</HeroCitationTitle>
+                        <HeroCitationBlock>
+                            <pre>
+                                <code>
+                                    <React.Fragment>{citation}</React.Fragment>
+                                </code>
+                            </pre>
+                        </HeroCitationBlock>
+                        <HeroFollowBlock>
+                            Follow me on <UDLLink href="https://twitter.com/SimonPrinceAI">Twitter</UDLLink> or <UDLLink
+                            href="https://www.linkedin.com/in/simon-prince-615bb9165/">LinkedIn</UDLLink> for updates.
+                        </HeroFollowBlock>
+                    </HeroColumn1>
+                    <HeroColumn2>
+                        <HeroImgWrap>
+                            <Img src={img} alt="book cover"/>
+                        </HeroImgWrap>
+                        <HeroLink href="https://github.com/udlbook/udlbook/releases/download/v2.05/UnderstandingDeepLearning_04_18_24_C.pdf">Download full pdf (18 Apr 2024)</HeroLink>
+                        <HeroDownloadsImg src="https://img.shields.io/github/downloads/udlbook/udlbook/total" alt="download stats shield"/>
+                        <HeroLink href="https://mitpress.mit.edu/9780262048644/understanding-deep-learning/">Buy the book</HeroLink>
+                        <HeroLink href="https://github.com/udlbook/udlbook/raw/main/UDL_Answer_Booklet_Students.pdf">Answers to selected questions</HeroLink>
+                        <HeroLink href="https://github.com/udlbook/udlbook/raw/main/UDL_Errata.pdf">Errata</HeroLink>
+                    </HeroColumn2>
+                </HeroRow>
+            </HeroContent>
+        </HeroContainer>
+    )
+}
+
+export default HeroSection

src/components/Instructors/InstructorsElements.js

@@ -0,0 +1,165 @@
+import styled from "styled-components";
+
+
+export const InstructorsContainer = styled.div` 
+    color: #fff;
+    /* background: #f9f9f9; */
+    background: ${({lightBg}) => (lightBg ? '#57c6d1': '#010606')};
+
+    @media screen and (max-width: 768px) {
+        padding: 100px 0;
+    }
+`
+
+export const InstructorsWrapper = styled.div`
+    display: grid ; 
+    z-index: 1;
+    width: 100% ; 
+    max-width: 1100px;
+    margin-right: auto;
+    margin-left: auto;
+    padding: 0 24px;
+    justify-content: center;
+`
+
+export const InstructorsRow = styled.div` 
+    display: grid; 
+    grid-auto-columns: minmax(auto, 1fr);
+    align-items: center;
+    grid-template-areas: ${({imgStart}) => (imgStart ? `'col2 col1'` : `'col1 col2'`)};
+
+    @media screen and (max-width: 768px){
+        grid-template-areas: ${({imgStart}) => (imgStart ? `'col1' 'col2'` : `'col1 col1' 'col2 col2'`)};
+    }
+`
+
+export const InstructorsRow2 = styled.div` 
+    display: grid; 
+    grid-auto-columns: minmax(auto, 1fr);
+    align-items: top;
+    grid-template-areas: ${({imgStart}) => (imgStart ? `'col2 col1'` : `'col1 col2'`)};
+
+    @media screen and (max-width: 768px){
+        grid-template-areas: ${({imgStart}) => (imgStart ? `'col1' 'col2'` : `'col1 col1' 'col2 col2'`)};
+    }
+`
+
+
+export const Column1 = styled.div` 
+    margin-bottom: 15px;
+    padding: 0 15px;
+    grid-area: col1;
+`
+
+export const Column2 = styled.div` 
+    margin-bottom: 15px;
+    padding: 0 15px;
+    grid-area: col2;
+`
+
+export const TextWrapper = styled.div`
+    max-width: 540px ; 
+    padding-top: 0;
+    padding-bottom: 0;
+`
+
+export const TopLine = styled.p` 
+    color: #773c23;
+    font-size: 16px;
+    line-height: 16px;
+    font-weight: 700;
+    letter-spacing: 1.4px;
+    text-transform: uppercase;
+    margin-bottom: 16px;
+`
+
+export const Heading= styled.h1` 
+
+    margin-bottom: 24px;
+    font-size: 48px;
+    line-height: 1.1;
+    font-weight: 600;
+    color: ${({lightText}) => (lightText ? '#f7f8fa' : '#010606')};
+    
+    @media screen and (max-width: 480px)
+    {
+        font-size: 32px;
+    }
+`
+
+
+export const Subtitle = styled.p` 
+    max-width: 440px;
+    margin-bottom: 35px;
+    font-size: 18px;
+    line-height: 24px;
+    color: ${({darkText})=> (darkText ? '#010606' : '#fff')};
+
+`
+
+export const BtnWrap = styled.div`
+    display: flex;
+    justify-content: flex-start;
+`
+
+export const ImgWrap = styled.div` 
+    max-width: 555px;
+    height: 100%;
+`
+
+export const Img = styled.img`
+  width: 100%;
+  margin-top: 0;
+  margin-right: 0;
+  margin-left: 10px;
+  padding-right: 0;
+`;
+
+
+export const InstructorsContent = styled.div` 
+    z-index: 3; 
+    width: 100% ; 
+    max-width: 1100px;
+    position: static;
+    padding: 8px 0px;
+    margin: 10px 0px;
+    display: flex;
+    flex-direction: column;
+    align-items: left ; 
+    list-style-position: inside;
+    @media screen and (max-width: 1050px) {
+        font-size: 12px;
+    }
+
+    @media screen and (max-width: 768px) {
+        font-size: 10px;
+    }
+`
+
+export const InstructorsLink = styled.a`
+    text-decoration: none;
+    color: #555;
+    font-weight: 300;
+    margin: 0 2px;
+    position: relative;
+
+    &:before{
+        position: absolute;
+        margin: 0 auto;
+        top: 100%;
+        left: 0;
+        width: 100%;
+        height: 2px;
+        background-color: #555;
+        content: '';
+        opacity: .3;
+        -webkit-transform: scaleX(1);
+        transition-property: opacity, -webkit-transform;
+        transition-duration: .3s;
+    }
+
+    &:hover:before {
+        opacity: 1;
+        -webkit-transform: scaleX(1.05);
+      }
+`

src/components/Instructors/index.js

@@ -0,0 +1,178 @@
+import React from 'react'
+import { ImgWrap, Img, InstructorsLink, InstructorsContainer, InstructorsContent, InstructorsRow2, InstructorsWrapper, InstructorsRow, Column1, Column2, TextWrapper, TopLine, Heading, Subtitle} from './InstructorsElements'
+
+// export const homeObjOne = {
+//     id: 'about',
+//     lightBg: false,
+//     lightText: true,
+//     lightTextDesc: true,
+//     topLine: 'Premium Bank',
+//     headline: 'Unlimited transactions with zero fees',
+//     description:
+//       'Get access to our exclusive app that allows you to send unlimited transactions without getting charged any fees',
+//     buttonLabel: 'Get Started',
+//     imgStart: false,
+//     img: require('../../images/svg-1.svg').default,
+//     alt: 'Car',
+//     dark: true,
+//     primary: true,
+//     darkText: false
+//   };
+
+import img from '../../images/instructor.svg'
+
+
+
+const InstructorsSection = () => {
+    return (
+        <>
+            <InstructorsContainer lightBg={true} id='Instructors'>
+                <InstructorsWrapper>
+                    <InstructorsRow imgStart={false}>
+                        <Column1>
+                            <TextWrapper>
+                                <TopLine>Instructors</TopLine>
+                                <Heading lightText={false}>Resources for instructors</Heading>
+                                <Subtitle darkText={true}>All the figures in vector and image formats, full slides for first twelve chapters, instructor answer booklet</Subtitle>
+                            </TextWrapper>
+                        </Column1>
+                        <Column2>
+                            <ImgWrap>
+                                <Img src={img} alt='Car'/>
+                            </ImgWrap>
+                        </Column2>
+                    </InstructorsRow>
+                    <InstructorsRow2>
+                        <Column1>
+                            <TopLine>Register</TopLine>
+                            <InstructorsLink href="https://mitpress.ublish.com/request?cri=15055">Register</InstructorsLink> with MIT Press for answer booklet.
+                            <InstructorsContent>
+
+                            </InstructorsContent>
+                           
+                            <TopLine>Full slides</TopLine>
+                            <InstructorsContent>
+                                Slides for 20 lecture undergraduate deep learning course:
+                            </InstructorsContent>
+                            <InstructorsContent>
+                                <ol>
+                                    <li>Introduction <InstructorsLink href="https://drive.google.com/uc?export=download&id=17RHb11BrydOvxSFNbRIomE1QKLVI087m">PPTX</InstructorsLink></li>
+                                    <li>Supervised Learning <InstructorsLink href="https://drive.google.com/uc?export=download&id=1491zkHULC7gDfqlV6cqUxyVYXZ-de-Ub">PPTX</InstructorsLink></li>
+                                    <li>Shallow Neural Networks <InstructorsLink href="https://drive.google.com/uc?export=download&id=1XkP1c9EhOBowla1rT1nnsDGMf2rZvrt7">PPTX</InstructorsLink></li>
+                                    <li>Deep Neural Networks <InstructorsLink href="https://drive.google.com/uc?export=download&id=1e2ejfZbbfMKLBv0v-tvBWBdI8gO3SSS1">PPTX</InstructorsLink></li>
+                                    <li>Loss Functions <InstructorsLink href="https://drive.google.com/uc?export=download&id=1fxQ_a1Q3eFPZ4kPqKbak6_emJK-JfnRH">PPTX</InstructorsLink></li>
+                                    <li>Fitting Models <InstructorsLink href="https://drive.google.com/uc?export=download&id=17QQ5ZzXBtR_uCNCUU1gPRWWRUeZN9exW">PPTX</InstructorsLink></li>
+                                    <li>Computing Gradients <InstructorsLink href="https://drive.google.com/uc?export=download&id=1hC8JUCOaFWiw3KGn0rm7nW6mEq242QDK">PPTX</InstructorsLink></li>
+                                    <li>Initialization <InstructorsLink href="https://drive.google.com/uc?export=download&id=1tSjCeAVg0JCeBcPgDJDbi7Gg43Qkh9_d">PPTX</InstructorsLink></li>
+                                    <li>Performance <InstructorsLink href="https://drive.google.com/uc?export=download&id=1RVZW3KjEs0vNSGx3B2fdizddlr6I0wLl">PPTX</InstructorsLink></li>
+                                    <li>Regularization <InstructorsLink href="https://drive.google.com/uc?export=download&id=1LTicIKPRPbZRkkg6qOr1DSuOB72axood">PPTX</InstructorsLink></li>
+                                    <li>Convolutional Networks <InstructorsLink href="https://drive.google.com/uc?export=download&id=1bGVuwAwrofzZdfvj267elIzkYMIvYFj0">PPTX</InstructorsLink></li>
+                                    <li>Image Generation <InstructorsLink href="https://drive.google.com/uc?export=download&id=14w31QqWRDix1GdUE-na0_E0kGKBhtKzs">PPTX</InstructorsLink></li>
+                                    <li>Transformers and LLMs <InstructorsLink href="https://drive.google.com/uc?export=download&id=1af6bTTjAbhDYfrDhboW7Fuv52Gk9ygKr">PPTX</InstructorsLink></li>
+                                </ol>
+                             </InstructorsContent>
+                        </Column1>
+                        <Column2>
+                            <TopLine>Figures</TopLine>
+                            <InstructorsContent>
+                            <ol>
+                                <li> Introduction: <InstructorsLink href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap1PDF.zip">PDF</InstructorsLink> / <InstructorsLink href="https://drive.google.com/uc?export=download&id=1udnl5pUOAc8DcAQ7HQwyzP9pwL95ynnv"> SVG</InstructorsLink> / <InstructorsLink href="https://docs.google.com/presentation/d/1IjTqIUvWCJc71b5vEJYte-Dwujcp7rvG/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PPTX </InstructorsLink></li>
+
+                            <li> Supervised learning: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap2PDF.zip">PDF</InstructorsLink> / <InstructorsLink
+                                        href="https://drive.google.com/uc?export=download&id=1VSxcU5y1qNFlmd3Lb3uOWyzILuOj1Dla"> SVG</InstructorsLink> / <InstructorsLink href="https://docs.google.com/presentation/d/1Br7R01ROtRWPlNhC_KOommeHAWMBpWtz/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PPTX</InstructorsLink></li>
+                                <li> Shallow neural networks: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap3PDF.zip">PDF</InstructorsLink> / <InstructorsLink
+                                        href="https://drive.google.com/uc?export=download&id=19kZFWlXhzN82Zx02ByMmSZOO4T41fmqI"> SVG</InstructorsLink> / <InstructorsLink href="https://docs.google.com/presentation/d/1e9M3jB5I9qZ4dCBY90Q3Hwft_i068QVQ/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PPTX</InstructorsLink></li>
+                                <li> Deep neural networks: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap4PDF.zip">PDF</InstructorsLink> / <InstructorsLink
+                                        href="https://drive.google.com/uc?export=download&id=1ojr0ebsOhzvS04ItAflX2cVmYqHQHZUa"> SVG</InstructorsLink>
+                                    /
+                                    <InstructorsLink href="https://docs.google.com/presentation/d/1LTSsmY4mMrJbqXVvoTOCkQwHrRKoYnJj/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PPTX</InstructorsLink></li>
+                                <li> Loss functions: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap5PDF.zip">PDF
+                                    </InstructorsLink> / <InstructorsLink href="https://drive.google.com/uc?export=download&id=17MJO7fiMpFZVqKeqXTbQ36AMpmR4GizZ">
+                                    SVG
+                                    </InstructorsLink> / <InstructorsLink
+                                        href="https://docs.google.com/presentation/d/1gcpC_3z9oRp87eMkoco-kdLD-MM54Puk/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PPTX</InstructorsLink></li>
+                                <li> Training models: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap6PDF.zip">PDF
+                                    </InstructorsLink> / <InstructorsLink href="https://drive.google.com/uc?export=download&id=1VPdhFRnCr9_idTrX0UdHKGAw2shUuwhK">
+                                    SVG
+                                    </InstructorsLink> / <InstructorsLink
+                                        href="https://docs.google.com/presentation/d/1AKoeggAFBl9yLC7X5tushAGzCCxmB7EY/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PPTX</InstructorsLink></li>
+                                <li> Gradients and initialization: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap7PDF.zip">PDF</InstructorsLink> / <InstructorsLink
+                                        href="https://drive.google.com/uc?export=download&id=1TTl4gvrTvNbegnml4CoGoKOOd6O8-PGs"> SVG</InstructorsLink> / <InstructorsLink href="https://docs.google.com/presentation/d/11zhB6PI-Dp6Ogmr4IcI6fbvbqNqLyYcz/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PPTX</InstructorsLink></li>
+                                <li> Measuring performance: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap8PDF.zip">PDF</InstructorsLink> / <InstructorsLink
+                                        href="https://drive.google.com/uc?export=download&id=19eQOnygd_l0DzgtJxXuYnWa4z7QKJrJx"> SVG</InstructorsLink> / <InstructorsLink href="https://docs.google.com/presentation/d/1SHRmJscDLUuQrG7tmysnScb3ZUAqVMZo/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PPTX</InstructorsLink></li>
+                                <li> Regularization: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap9PDF.zip">PDF
+                                    </InstructorsLink> / <InstructorsLink href="https://drive.google.com/uc?export=download&id=1LprgnUGL7xAM9-jlGZC9LhMPeefjY0r0">
+                                    SVG
+                                    </InstructorsLink> / <InstructorsLink
+                                        href="https://docs.google.com/presentation/d/1VwIfvjpdfTny6sEfu4ZETwCnw6m8Eg-5/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PPTX</InstructorsLink></li>
+                                <li> Convolutional networks: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap10PDF.zip">PDF</InstructorsLink> / <InstructorsLink
+                                        href="https://drive.google.com/uc?export=download&id=1-Wb3VzaSvVeRzoUzJbI2JjZE0uwqupM9"> SVG</InstructorsLink> / <InstructorsLink href="https://docs.google.com/presentation/d/1MtfKBC4Y9hWwGqeP6DVwUNbi1j5ncQCg/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PPTX</InstructorsLink></li>
+                                <li> Residual networks: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap11PDF.zip">PDF</InstructorsLink> / <InstructorsLink
+                                        href="https://drive.google.com/uc?export=download&id=1Mr58jzEVseUAfNYbGWCQyDtEDwvfHRi1"> SVG</InstructorsLink> / <InstructorsLink href="https://docs.google.com/presentation/d/1saY8Faz0KTKAAifUrbkQdLA2qkyEjOPI/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PPTX</InstructorsLink></li>
+                                <li> Transformers: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap12PDF.zip">PDF</InstructorsLink> / <InstructorsLink href="https://drive.google.com/uc?export=download&id=1txzOVNf8-jH4UfJ6SLnrtOfPd1Q3ebzd">
+                                    SVG</InstructorsLink> / <InstructorsLink
+                                        href="https://docs.google.com/presentation/d/1GVNvYWa0WJA6oKg89qZre-UZEhABfm0l/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PPTX</InstructorsLink></li>
+                                <li> Graph neural networks: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap13PDF.zip">PDF</InstructorsLink> / <InstructorsLink
+                                        href="https://drive.google.com/uc?export=download&id=1lQIV6nRp6LVfaMgpGFhuwEXG-lTEaAwe"> SVG</InstructorsLink> / <InstructorsLink href="https://docs.google.com/presentation/d/1YwF3U82c1mQ74c1WqHVTzLZ0j7GgKaWP/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PPTX</InstructorsLink></li>
+                                <li> Unsupervised learning: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap14PDF.zip">PDF</InstructorsLink> / <InstructorsLink
+                                        href="https://drive.google.com/uc?export=download&id=1aMbI6iCuUvOywqk5pBOmppJu1L1anqsM"> SVG</InstructorsLink> / <InstructorsLink href="https://docs.google.com/presentation/d/1A-lBGv3NHl4L32NvfFgy1EKeSwY-0UeB/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">
+                                        PPTX</InstructorsLink></li>
+                                <li> GANs: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap15PDF.zip">PDF</InstructorsLink> / <InstructorsLink
+                                        href="https://drive.google.com/uc?export=download&id=1EErnlZCOlXc3HK7m83T2Jh_0NzIUHvtL"> SVG</InstructorsLink> / <InstructorsLink href="https://docs.google.com/presentation/d/10Ernk41ShOTf4IYkMD-l4dJfKATkXH4w/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PPTX</InstructorsLink></li>
+                                <li> Normalizing flows: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap16PDF.zip">PDF</InstructorsLink> / <InstructorsLink
+                                        href="https://drive.google.com/uc?export=download&id=1SNtNIY7khlHQYMtaOH-FosSH3kWwL4b7"> SVG</InstructorsLink> / <InstructorsLink href="https://docs.google.com/presentation/d/1nLLzqb9pdfF_h6i1HUDSyp7kSMIkSUUA/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PPTX</InstructorsLink></li>
+                                <li> Variational autoencoders: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap17PDF.zip">PDF</InstructorsLink> / <InstructorsLink
+                                        href="https://drive.google.com/uc?export=download&id=1B9bxtmdugwtg-b7Y4AdQKAIEVWxjx8l3"> SVG</InstructorsLink> / <InstructorsLink href="https://docs.google.com/presentation/d/1lQE4Bu7-LgvV2VlJOt_4dQT-kusYl7Vo/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PPTX</InstructorsLink></li>
+                                <li> Diffusion models: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap18PDF.zip">PDF</InstructorsLink> / <InstructorsLink
+                                        href="https://drive.google.com/uc?export=download&id=1A-pIGl4PxjVMYOKAUG3aT4a8wD3G-q_r"> SVG</InstructorsLink> / 
+                                    <InstructorsLink href="https://docs.google.com/presentation/d/1x_ufIBtVPzWUvRieKMkpw5SdRjXWwdfR/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">
+                                    PPTX</InstructorsLink></li>
+                                <li> Deep reinforcement learning: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap19PDF.zip">PDF</InstructorsLink> / <InstructorsLink
+                                        href="https://drive.google.com/uc?export=download&id=1a5WUoF7jeSgwC_PVdckJi1Gny46fCqh0"> SVG</InstructorsLink> / <InstructorsLink href="https://docs.google.com/presentation/d/1TnYmVbFNhmMFetbjyfXGmkxp1EHauMqr/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">
+                                        PPTX </InstructorsLink></li>
+                                <li> Why does deep learning work?: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap20PDF.zip">PDF</InstructorsLink> / <InstructorsLink
+                                        href="https://drive.google.com/uc?export=download&id=1M2d0DHEgddAQoIedKSDTTt7m1ZdmBLQ3"> SVG</InstructorsLink> / <InstructorsLink href="https://docs.google.com/presentation/d/1coxF4IsrCzDTLrNjRagHvqB_FBy10miA/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">
+                                        PPTX</InstructorsLink></li>
+                                <li> Deep learning and ethics: <InstructorsLink
+                                        href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap21PDF.zip">PDF</InstructorsLink> / <InstructorsLink
+                                        href="https://drive.google.com/uc?export=download&id=1jixmFfwmZkW_UVYzcxmDcMsdFFtnZ0bU">SVG</InstructorsLink> / <InstructorsLink
+                                            href="https://docs.google.com/presentation/d/1EtfzanZYILvi9_-Idm28zD94I_6OrN9R/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PPTX</InstructorsLink></li>
+                                <li> Appendices - <InstructorsLink href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLAppendixPDF.zip">PDF</InstructorsLink> / <InstructorsLink href="https://drive.google.com/uc?export=download&id=1k2j7hMN40ISPSg9skFYWFL3oZT7r8v-l">
+                                    SVG</InstructorsLink> / <InstructorsLink
+                                        href="https://docs.google.com/presentation/d/1_2cJHRnsoQQHst0rwZssv-XH4o5SEHks/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PPTX</InstructorsLink></li>
+                            </ol>
+                            </InstructorsContent>
+                            <InstructorsLink href="https://drive.google.com/file/d/1T_MXXVR4AfyMnlEFI-UVDh--FXI5deAp/view?usp=sharing">Instructions</InstructorsLink> for editing equations in figures.
+
+                            <InstructorsContent>
+                                
+                            </InstructorsContent>
+                        </Column2>
+                    </InstructorsRow2>
+
+                </InstructorsWrapper>
+            </InstructorsContainer>
+        </>
+    )
+}
+
+export default InstructorsSection

src/components/Media/MediaElements.js

@@ -0,0 +1,183 @@
+import styled from "styled-components";
+
+
+export const MediaContainer = styled.div` 
+    color: #fff;
+    /* background: #f9f9f9; */
+    background: ${({lightBg}) => (lightBg ? '#f9f9f9': '#010606')};
+
+    @media screen and (max-width: 768px) {
+        padding: 100px 0;
+    }
+`
+
+export const MediaWrapper = styled.div`
+    display: grid ; 
+    z-index: 1;
+    width: 100% ; 
+    max-width: 1100px;
+    margin-right: auto;
+    margin-left: auto;
+    padding: 0 24px;
+    justify-content: center;
+`
+
+export const MediaRow = styled.div` 
+    display: grid; 
+    grid-auto-columns: minmax(auto, 1fr);
+    align-items: center;
+    grid-template-areas: ${({imgStart}) => (imgStart ? `'col2 col1'` : `'col1 col2'`)};
+
+    @media screen and (max-width: 768px){
+        grid-template-areas: ${({imgStart}) => (imgStart ? `'col1' 'col2'` : `'col1 col1' 'col2 col2'`)};
+    }
+`
+
+export const Column1 = styled.div` 
+    margin-bottom: 15px;
+    padding: 0 15px;
+    grid-area: col1;
+`
+
+export const Column2 = styled.div` 
+    margin-bottom: 15px;
+    padding: 0 15px;
+    grid-area: col2;
+`
+
+export const TextWrapper = styled.div`
+    max-width: 540px ; 
+    padding-top: 0;
+    padding-bottom: 0;
+`
+
+export const TopLine = styled.p` 
+    color: #57c6d1;
+    font-size: 16px;
+    line-height: 16px;
+    font-weight: 700;
+    letter-spacing: 1.4px;
+    text-transform: uppercase;
+    margin-bottom: 16px;
+`
+
+export const Heading= styled.h1` 
+
+    margin-bottom: 24px;
+    font-size: 48px;
+    line-height: 1.1;
+    font-weight: 600;
+    color: ${({lightText}) => (lightText ? '#f7f8fa' : '#010606')};
+    
+    @media screen and (max-width: 480px)
+    {
+        font-size: 32px;
+    }
+`
+
+
+export const Subtitle = styled.p` 
+    max-width: 440px;
+    margin-bottom: 35px;
+    font-size: 18px;
+    line-height: 24px;
+    color: ${({darkText})=> (darkText ? '#010606' : '#fff')};
+
+`
+
+export const BtnWrap = styled.div`
+    display: flex;
+    justify-content: flex-start;
+`
+
+export const ImgWrap = styled.div` 
+    max-width: 555px;
+    height: 100%;
+`
+
+export const Img = styled.img`
+  width: 100%;
+  margin-top: 0;
+  margin-right: 0;
+  margin-left: 10px;
+  padding-right: 0;
+`;
+
+
+export const MediaTextBlock = styled.div`
+@media screen and (max-width: 768px) {
+    font-size: 24px;
+}
+
+@media screen and (max-width: 480px) {
+    font-size: 18px;
+}
+`
+
+export const MediaContent = styled.div` 
+    z-index: 3; 
+    width: 100% ; 
+    max-width: 1100px;
+    position: static;
+    padding: 8px 0px;
+    margin: 10px 0px;
+    display: flex;
+    flex-direction: column;
+    align-items: left ; 
+    list-style-position: inside;
+    @media screen and (max-width: 768px) {
+        font-size: 14px;
+    }
+
+`
+
+export const MediaRow2 = styled.div` 
+    display: grid; 
+    grid-auto-columns: minmax(auto, 1fr);
+    align-items: top;
+    grid-template-areas: ${({imgStart}) => (imgStart ? `'col2 col1'` : `'col1 col2'`)};
+
+    @media screen and (max-width: 768px){
+        grid-template-areas: ${({imgStart}) => (imgStart ? `'col1' 'col2'` : `'col1 col1' 'col2 col2'`)};
+    }
+`
+
+export const VideoFrame=styled.div`
+    width: 560px ; 
+    height: 315px ; 
+    @media screen and (max-width: 1050px) {
+        width: 280px ; 
+        height: 157px ; 
+    }
+
+
+`
+
+
+export const MediaLink = styled.a`
+    text-decoration: none;
+    color: #57c6d1;
+    font-weight: 300;
+    margin: 0 2px;
+    position: relative;
+
+    &:before{
+        position: absolute;
+        margin: 0 auto;
+        top: 100%;
+        left: 0;
+        width: 100%;
+        height: 2px;
+        background-color: #57c6d1;;
+        content: '';
+        opacity: .3;
+        -webkit-transform: scaleX(1);
+        transition-property: opacity, -webkit-transform;
+        transition-duration: .3s;
+    }
+
+    &:hover:before {
+        opacity: 1;
+        -webkit-transform: scaleX(1.05);
+      }
+`

src/components/Media/index.js

@@ -0,0 +1,90 @@
+import React from 'react'
+import { ImgWrap, Img, MediaLink, MediaContainer, MediaContent, MediaWrapper, VideoFrame, MediaRow, MediaRow2, Column1, Column2, TextWrapper, TopLine, Heading, Subtitle} from './MediaElements'
+
+// export const homeObjOne = {
+//     id: 'about',
+//     lightBg: false,
+//     lightText: true,
+//     lightTextDesc: true,
+//     topLine: 'Premium Bank',
+//     headline: 'Unlimited transactions with zero fees',
+//     description:
+//       'Get access to our exclusive app that allows you to send unlimited transactions without getting charged any fees',
+//     buttonLabel: 'Get Started',
+//     imgStart: false,
+//     img: require('../../images/svg-1.svg').default,
+//     alt: 'Car',
+//     dark: true,
+//     primary: true,
+//     darkText: false
+//   };
+
+import img from '../../images/media.svg'
+
+
+
+const MediaSection = () => {
+    return (
+        <>
+            <MediaContainer lightBg={false} id='Media'>
+                <MediaWrapper>
+                    <MediaRow imgStart={true}>
+                        <Column1>
+                            <TextWrapper>
+                                <TopLine>Media</TopLine>
+                                <Heading lightText={true}> Reviews, videos, podcasts, interviews</Heading>
+                                <Subtitle darkText={false}>Various resources connected to the book</Subtitle>                            
+                            </TextWrapper>
+                        </Column1>
+                        <Column2>
+                            <ImgWrap>
+                                <Img src={img} alt='Car'/>
+                            </ImgWrap>
+                        </Column2>
+                    </MediaRow>
+                    <MediaRow>
+                        <Column1>
+                            Machine learning street talk podcast
+                            <VideoFrame>
+                                <iframe width="100%" height="100%" 
+                                    src="https://www.youtube.com/embed/sJXn4Cl4oww?si=Lm_hQPqj0RXy-75H&amp;controls=0" 
+                                    title="YouTube video player" frameborder="2" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen>
+                                </iframe>
+                            </VideoFrame>
+                        </Column1>
+                        <Column2>
+                            Deeper insights podcast 
+                            <VideoFrame>
+                                <iframe width="100%" height="100%" src="https://www.youtube.com/embed/nQf4o9TDSHI?si=uMk66zLD7uhuSnQ1&amp;controls=0" title="YouTube video player" frameborder="2" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
+                            </VideoFrame>
+                        </Column2>
+                    </MediaRow>
+                    <MediaRow2>
+                        <Column1>
+                            <TopLine>Reviews</TopLine>
+                            <MediaContent>
+                                <ul>
+                                    <li> Amazon <MediaLink href="https://www.amazon.com/Understanding-Deep-Learning-Simon-Prince-ebook/product-reviews/B0BXKH8XY6/">reviews</MediaLink></li>          
+                                    <li>Goodreads <MediaLink href="https://www.goodreads.com/book/show/123239819-understanding-deep-learning?">reviews </MediaLink></li>
+                                    <li>Book <MediaLink href="https://medium.com/@vishalvignesh/udl-book-review-the-new-deep-learning-textbook-youll-want-to-finish-69e1557b018d">review</MediaLink> by Vishal V.</li>
+                                </ul>
+                            </MediaContent>
+                        </Column1>
+                        <Column2>
+                            <TopLine>Interviews</TopLine>
+                            <MediaContent>
+                                <ul>
+                                    <li>Borealis AI <MediaLink href="https://www.borealisai.com/news/understanding-deep-learning/">interview</MediaLink></li>
+                                    <li>Shepherd ML book <MediaLink href="https://shepherd.com/best-books/machine-learning-and-deep-neural-networks">recommendations</MediaLink></li>
+                                </ul>
+                            </MediaContent>
+                        </Column2>
+                    </MediaRow2>
+
+                </MediaWrapper>
+            </MediaContainer>
+        </>
+    )
+}
+
+export default MediaSection

src/components/More/MoreElements.js

@@ -0,0 +1,187 @@
+import styled from "styled-components";
+
+
+export const MoreContainer = styled.div` 
+    color: #fff;
+    /* background: #f9f9f9; */
+    background: ${({lightBg}) => (lightBg ? '#57c6d1': '#010606')};
+
+    @media screen and (max-width: 768px) {
+        padding: 100px 0;
+    }
+`
+
+export const MoreWrapper = styled.div`
+    display: grid ; 
+    z-index: 1;
+    // height: 1050px ;  
+    width: 100% ; 
+    max-width: 1100px;
+    margin-right: auto;
+    margin-left: auto;
+    padding: 0 24px;
+    justify-content: center;
+`
+
+export const MoreRow = styled.div` 
+    display: grid; 
+    grid-auto-columns: minmax(auto, 1fr);
+    align-items: center;
+    grid-template-areas: ${({imgStart}) => (imgStart ? `'col2 col1'` : `'col1 col2'`)};
+
+    @media screen and (max-width: 768px){
+        grid-template-areas: ${({imgStart}) => (imgStart ? `'col1' 'col2'` : `'col1 col1' 'col2 col2'`)};
+    }
+`
+
+export const MoreRow2 = styled.div` 
+    display: grid; 
+    grid-auto-columns: minmax(auto, 1fr);
+    align-items: top;
+    grid-template-areas: ${({imgStart}) => (imgStart ? `'col2 col1'` : `'col1 col2'`)};
+
+    @media screen and (max-width: 768px){
+        grid-template-areas: ${({imgStart}) => (imgStart ? `'col1' 'col2'` : `'col1 col1' 'col2 col2'`)};
+    }
+`
+
+
+export const Column1 = styled.div` 
+    margin-bottom: 15px;
+    padding: 0 15px;
+    grid-area: col1;
+`
+
+export const Column2 = styled.div` 
+    margin-bottom: 15px;
+    padding: 0 15px;
+    grid-area: col2;
+`
+
+export const TextWrapper = styled.div`
+    max-width: 540px ; 
+    padding-top: 0;
+    padding-bottom: 0;
+`
+
+export const TopLine = styled.p` 
+    color: #773c23;
+    font-size: 16px;
+    line-height: 16px;
+    font-weight: 700;
+    letter-spacing: 1.4px;
+    text-transform: uppercase;
+    margin-bottom: 12px;
+    margin-top: 16px ; 
+`
+
+export const Heading= styled.h1` 
+
+    margin-bottom: 24px;
+    font-size: 48px;
+    line-height: 1.1;
+    font-weight: 600;
+    color: ${({lightText}) => (lightText ? '#f7f8fa' : '#010606')};
+    
+    @media screen and (max-width: 480px)
+    {
+        font-size: 32px;
+    }
+`
+
+
+export const Subtitle = styled.p` 
+    max-width: 440px;
+    margin-bottom: 35px;
+    font-size: 18px;
+    line-height: 24px;
+    color: ${({darkText})=> (darkText ? '#010606' : '#fff')};
+
+`
+
+export const BtnWrap = styled.div`
+    display: flex;
+    justify-content: flex-start;
+`
+
+export const ImgWrap = styled.div` 
+    max-width: 555px;
+    height: 100%;
+`
+
+export const Img = styled.img`
+  width: 100%;
+  margin-top: 0;
+  margin-right: 0;
+  margin-left: 10px;
+  padding-right: 0;
+`;
+
+
+export const MoreContent = styled.div` 
+    z-index: 3; 
+    width: 100% ; 
+    max-width: 1100px;
+    position: static;
+    padding: 8px 0px;
+    margin: 10px 0px;
+    display: flex;
+    flex-direction: column;
+    align-items: left ; 
+    list-style-position: inside;
+`
+
+export const MoreOuterList = styled.ul`
+    // list-style:none;
+    list-style-position: inside;
+    margin:0;
+    @media screen and (max-width: 768px) {
+        font-size: 14px;
+    }
+
+`
+
+export const MoreInnerList = styled.ul`
+    list-style-position: inside;
+    @media screen and (max-width: 768px) {
+        font-size: 12px;
+    }
+
+`
+
+export const MoreInnerP = styled.p`
+    padding-left: 18px;
+    padding-bottom: 10px ; 
+    padding-top: 3px ; 
+    font-size:14px;
+    color: #fff
+`
+
+
+export const MoreLink = styled.a`
+    text-decoration: none;
+    color: #555;
+    font-weight: 300;
+    margin: 0 2px;
+    position: relative;
+
+    &:before{
+        position: absolute;
+        margin: 0 auto;
+        top: 100%;
+        left: 0;
+        width: 100%;
+        height: 2px;
+        background-color: #555;
+        content: '';
+        opacity: .3;
+        -webkit-transform: scaleX(1);
+        transition-property: opacity, -webkit-transform;
+        transition-duration: .3s;
+    }
+
+    &:hover:before {
+        opacity: 1;
+        -webkit-transform: scaleX(1.05);
+      }
+`

src/components/More/index.js

@@ -0,0 +1,750 @@
+import React from 'react'
+import { ImgWrap, Img, MoreContainer, MoreLink, MoreRow2, MoreWrapper, MoreRow, Column1, Column2, TextWrapper, TopLine, Heading, Subtitle, MoreOuterList, MoreInnerList, MoreInnerP} from './MoreElements'
+import img from '../../images/more.svg'
+
+
+const MoreSection = () => {
+    return (
+        <>
+            <MoreContainer lightBg={true} id='More'>
+                <MoreWrapper> 
+                    <MoreRow imgStart={false}>
+                        <Column1>
+                            <TextWrapper>
+                                <TopLine>More</TopLine>
+                                <Heading lightText={false}>Further reading</Heading>
+                                <Subtitle darkText={true}>Other articles, blogs, and books that I have written. Most in a similar style and using the same notation as Understanding Deep Learning. </Subtitle>
+                            </TextWrapper>
+                        </Column1>
+                        <Column2>
+                            <ImgWrap>
+                                <Img src={img} alt='Car'/>
+                            </ImgWrap>
+                        </Column2>
+                    </MoreRow>
+                    <MoreRow2>
+                     
+                        <Column1>
+                            <TopLine>Book</TopLine>
+                                <MoreOuterList>
+                                    <li>
+                                        <MoreLink href="http://computervisionmodels.com" target="_blank" rel="noreferrer">Computer vision: models, learning, and inference</MoreLink>
+                                        <MoreInnerP>
+                                            <MoreInnerList>
+                                                <li> 2012 book published with CUP </li>
+                                                <li> Focused on probabilistic models </li>
+                                                <li> Pre-"deep learning" </li>
+                                                <li> Lots of ML content</li>
+                                                <li> Individual chapters available below</li>
+                                            </MoreInnerList> 
+                                        </MoreInnerP>
+                                    </li>
+                                </MoreOuterList>
+
+                            <TopLine>Transformers & LLMs</TopLine>
+                            <MoreOuterList>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/research-blogs/a-high-level-overview-of-large-language-models/" target="_blank"  rel="noreferrer">Intro to LLMs</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> What is an LLM?</li>
+                                            <li> Pretraining</li>
+                                            <li> Instruction fine-tuning</li>
+                                            <li> Reinforcement learning from human feedback</li>
+                                            <li> Notable LLMs</li>
+                                            <li> LLMs without training from scratch</li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial-14-transformers-i-introduction/" target="_blank"  rel="noreferrer">Transformers I</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Dot-Product self-attention </li>
+                                            <li> Scaled dot-product self-attention </li>
+                                            <li> Position encoding</li>
+                                            <li> Multiple heads </li>
+                                            <li> Transformer block </li>
+                                            <li> Encoders </li>
+                                            <li> Decoders </li>
+                                            <li> Encoder-Decoders </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial-16-transformers-ii-extensions/" target="_blank"  rel="noreferrer">Transformers II</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Sinusoidal position embeddings </li>
+                                            <li> Learned position embeddings </li>
+                                            <li> Relatives vs. absolute position embeddings</li>
+                                            <li> Extending transformers to longer sequences </li>
+                                            <li> Reducing attention matrix size </li>
+                                            <li> Making attention matrix sparse </li>
+                                            <li> Kernelizing attention computation </li>
+                                            <li> Attention as an RNN</li>
+                                            <li> Attention as a hypernetwork </li>
+                                            <li> Attention as a routing network </li>
+                                            <li> Attention and graphs </li>
+                                            <li> Attention and convolutions </li>
+                                            <li> Attention and gating </li>
+                                            <li> Attention and memory retrieval </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial-17-transformers-iii-training/" target="_blank"  rel="noreferrer">Transformers III</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Tricks for training transformers </li>
+                                            <li> Why are these tricks required? </li>
+                                            <li> Removing layer normalization</li>
+                                            <li> Balancing residual dependencies </li>
+                                            <li> Reducing optimizer variance </li>
+                                            <li> How to train deeper transformers on small datasets </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/research-blogs/training-and-fine-tuning-large-language-models/" target="_blank"  rel="noreferrer">Training and fine-tuning LLMs</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Large language models </li>
+                                            <li> Pretraining </li>
+                                            <li> Supervised fine tuning</li>
+                                            <li> Reinforcement learning from human feedback </li>
+                                            <li> Direct preference optimization</li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/research-blogs/speeding-up-inference-in-transformers/" target="_blank"  rel="noreferrer">Speeding up inference in LLMs</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Problems with transformers</li>
+                                            <li> Attention-free transformers </li>
+                                            <li> Complexity</li>
+                                            <li> RWKV </li>
+                                            <li> Linear transformers and performers</li>
+                                            <li> Retentive network</li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                            </MoreOuterList>
+
+                            <TopLine>Math for machine learning</TopLine>
+                            <MoreOuterList>
+                                <li>
+                                    <MoreLink href="https://drive.google.com/file/d/1j2v2n6STPnblOCZ1_GBcVAZrsYkjPYwR/view?usp=sharing" target="_blank"  rel="noreferrer">Linear algebra</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Vectors and matrices </li>
+                                            <li> Determinant and trace </li>
+                                            <li> Orthogonal matrices </li>
+                                            <li> Null space </li>
+                                            <li> Linear transformations </li>
+                                            <li> Singular value decomposition </li>
+                                            <li> Least squares problems </li>
+                                            <li> Principal direction problems </li>
+                                            <li> Inversion of block matrices</li>
+                                            <li> Schur complement identity</li>
+                                            <li> Sherman-Morrison-Woodbury</li>
+                                            <li> Matrix determinant lemma</li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://drive.google.com/file/d/1cmxXneW122-hcfmMRjEE-n5C9T2YvuQX/view?usp=sharing" target="_blank"  rel="noreferrer">Introduction to probability</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Random variables </li>
+                                            <li> Joint probability </li>
+                                            <li> Marginal probability </li>
+                                            <li> Conditional probability </li>
+                                            <li> Bayes' rule </li>
+                                            <li> Independence </li>
+                                            <li> Expectation </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://drive.google.com/file/d/1GI3eZNB1CjTqYHLyuRhCV215rwqANVOx/view?usp=sharing" target="_blank"  rel="noreferrer">Probability distributions</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Bernouilli distribution </li>
+                                            <li> Beta distribution</li>
+                                            <li> Categorical distribution </li>
+                                            <li> Dirichlet distribution</li>
+                                            <li> Univariate normal distribution </li>
+                                            <li> Normal inverse-scaled gamma distribution </li>
+                                            <li> Multivariate normal distribution </li>
+                                            <li> Normal inverse Wishart distribution </li>
+                                            <li> Conjugacy </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://drive.google.com/file/d/1DZ4rCmC7AZ8PFc51PiMUIkBO-xqKT_CG/view?usp=sharing" target="_blank"  rel="noreferrer">Fitting probability distributions</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Maximum likelihood </li>
+                                            <li> Maximum a posteriori </li>
+                                            <li> Bayesian approach </li>
+                                            <li> Example: fitting normal </li>
+                                            <li> Example: fitting categorical </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://drive.google.com/file/d/1CTfmsN-HJWZBRj8lY0ZhgHEbPCmYXWnA/view?usp=sharing" target="_blank"  rel="noreferrer">The normal distribution</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Types of covariance matrix </li>
+                                            <li> Decomposition of covariance </li>
+                                            <li> Linear transformations </li>
+                                            <li> Marginal distributions </li>
+                                            <li> Conditional distributions </li>
+                                            <li> Product of two normals </li>
+                                            <li> Change of variable formula </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                            </MoreOuterList>
+
+                            <TopLine>Optimization</TopLine>
+                            <MoreOuterList>
+                                <li>
+                                    <MoreLink href="https://drive.google.com/file/d/1IoOSfJ0ku89aVyM9qygPl4MVnAhMEbAZ/view?usp=sharing" target="_blank"  rel="noreferrer">Gradient-based optimization</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Convexity </li>
+                                            <li> Steepest descent </li>
+                                            <li> Newton's method </li>
+                                            <li> Gauss-Newton method </li>
+                                            <li> Line search </li>
+                                            <li> Reparameterization </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial-8-bayesian-optimization/" target="_blank"  rel="noreferrer">Bayesian optimization</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Gaussian processes </li>
+                                            <li> Acquisition functions </li>
+                                            <li> Incorporating noise</li>
+                                            <li> Kernel choice </li>
+                                            <li> Learning GP parameters </li>
+                                            <li> Tips, tricks, and limitations  </li>
+                                            <li> Beta-Bernoulli bandit  </li>
+                                            <li> Random forests for BO </li>
+                                            <li> Tree-Parzen estimators </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial-9-sat-solvers-i-introduction-and-applications/" target="_blank"  rel="noreferrer">SAT Solvers I</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Boolean logic and satisfiability </li>
+                                            <li> Conjunctive normal form </li>
+                                            <li> The Tseitin transformation </li>
+                                            <li> SAT and related problems </li>
+                                            <li> SAT constructions </li>
+                                            <li> Graph coloring and scheduling </li>
+                                            <li> Fitting binary neural networks</li>
+                                            <li> Fitting decision trees</li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial-10-sat-solvers-ii-algorithms/" target="_blank"  rel="noreferrer">SAT Solvers II</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Conditioning </li>
+                                            <li> Resolution </li>
+                                            <li> Solving 2-SAT by unit propagation </li>
+                                            <li> Directional resolution </li>
+                                            <li> SAT as binary search </li>
+                                            <li> DPLL </li>
+                                            <li> Conflict driven clause learning</li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial-11-sat-solvers-iii-factor-graphs-and-smt-solvers/" target="_blank"  rel="noreferrer">SAT Solvers III</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Satisfiability vs. problem size </li>
+                                            <li> Factor graph representation </li>
+                                            <li> Max product / sum product for SAT </li>
+                                            <li> Survey propagation </li>
+                                            <li> SAT with non-binary variables </li>
+                                            <li> SMT solvers </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                            </MoreOuterList>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial-11-sat-solvers-iii-factor-graphs-and-smt-solvers/" target="_blank"  rel="noreferrer">SAT Solvers III</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Satisfiability vs. problem size </li>
+                                            <li> Factor graph representation </li>
+                                            <li> Max product / sum product for SAT </li>
+                                            <li> Survey propagation </li>
+                                            <li> SAT with non-binary variables </li>
+                                            <li> SMT solvers </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+
+                            <TopLine>Computer vision</TopLine>
+                                <MoreOuterList>
+                                    <li>
+                                        <MoreLink href="https://drive.google.com/file/d/1r3V1GC5grhPF2pD91izuE0hTrTUEpQ9I/view?usp=sharing" target="_blank" rel="noreferrer">Image Processing</MoreLink>
+                                        <MoreInnerP>
+                                            <MoreInnerList>
+                                                <li> Whitening </li>
+                                                <li> Histogram equalization </li>
+                                                <li> Filtering </li>
+                                                <li> Edges and corners </li>
+                                                <li> Dimensionality reduction </li>
+                                            </MoreInnerList> 
+                                        </MoreInnerP>
+                                    </li>
+                                    <li>
+                                        <MoreLink href="https://drive.google.com/file/d/1dbMBE13MWcd84dEGjYeWsC6eXouoC0xn/view?usp=sharing" target="_blank" rel="noreferrer">Pinhole camera</MoreLink>
+                                        <MoreInnerP>
+                                            <MoreInnerList> 
+                                                <li> Pinhole camera model </li>
+                                                <li> Radial distortion </li>
+                                                <li> Homogeneous coordinates </li>
+                                                <li> Learning extrinsic parameters </li>
+                                                <li> Learning intrinsic parameters </li>
+                                                <li> Inferring three-dimensional world points </li>
+                                            </MoreInnerList>
+                                        </MoreInnerP>
+                                    </li>
+                                <li>
+                                    <MoreLink href="https://drive.google.com/file/d/1UArrb1ovqvZHbv90MufkW372r__ZZACQ/view?usp=sharing" target="_blank" rel="noreferrer">Geometric transformations</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Euclidean, similarity, affine, projective transformations </li>
+                                            <li> Fitting transformation models </li>
+                                            <li> Inference in transformation models </li>
+                                            <li> Three geometric problems for planes </li>
+                                            <li> Transformations between images </li>
+                                            <li> Robust learning of transformations </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://drive.google.com/file/d/1RqUoc7kvK8vqZF1NVuw7bIex9v4_QlSx/view?usp=sharing" target="_blank" rel="noreferrer">Multiple cameras</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Two view geometry </li>
+                                            <li> The essential matrix </li>
+                                            <li> The fundamental matrix </li>
+                                            <li> Two-view reconstruction pipeline </li>
+                                            <li> Rectification </li>
+                                            <li> Multiview reconstruction </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                            </MoreOuterList>  
+
+                            <TopLine>Reinforcement learning</TopLine>
+                            <MoreOuterList>
+                                <li>
+                                    <MoreLink href="https://arxiv.org/abs/2307.05979" target="_blank"  rel="noreferrer">Transformers in RL</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Challenges in RL</li>
+                                            <li> Advantages of transformers for RL</li>
+                                            <li> Representation learning</li>
+                                            <li> Transition function learning</li>
+                                            <li> Reward learning </li>
+                                            <li> Policy learning </li>
+                                            <li> Training strategy </li>
+                                            <li> Interpretability </li>
+                                            <li> Applications </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                            </MoreOuterList>
+                        </Column1>
+
+                        {/* ########################################### */}
+
+                        <Column2>
+                            <TopLine>AI Theory</TopLine>
+                            <MoreOuterList>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/research-blogs/gradient-flow/" target="_blank"  rel="noreferrer">Gradient flow</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Gradient flow </li>
+                                            <li> Evolution of residual </li>
+                                            <li> Evolution of parameters </li>
+                                            <li> Evolution of model predictions </li>
+                                            <li> Evolution of prediction covariance </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                <MoreLink href="https://www.borealisai.com/research-blogs/the-neural-tangent-kernel/" target="_blank"  rel="noreferrer">Neural tangent kernel</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Infinite width neural networks </li>
+                                            <li> Training dynamics </li>
+                                            <li> Empirical NTK for shallow network</li>
+                                            <li> Analytical NTK for shallow network </li>
+                                            <li> Empirical NTK for ddep network </li>
+                                            <li> Analtical NTK for deep network</li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                            </MoreOuterList>
+
+                            <TopLine>Temporal models</TopLine>
+                            <MoreOuterList>
+                                <li>
+                                    <MoreLink href="https://drive.google.com/file/d/1rrzGNyZDjXQ3_9ZqCGDmRMM3GYtHSBvj/view?usp=sharing" target="_blank"  rel="noreferrer">Temporal models</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Kalman filter </li>
+                                            <li> Smoothing </li>
+                                            <li> Extended Kalman filter </li>
+                                            <li> Unscented Kalman filter </li>
+                                            <li> Particle filtering </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                            </MoreOuterList>
+
+                            <TopLine> Unsupervised learning</TopLine>
+                            <MoreOuterList>
+                                <li>
+                                    <MoreLink href="https://drive.google.com/file/d/1BrPHxAuyz28hhz_FtbO0A1cWYdMs2_h8/view?usp=sharing" target="_blank"  rel="noreferrer">Modeling complex data densities</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Hidden variables </li>
+                                            <li> Expectation maximization </li>
+                                            <li> Mixture of Gaussians </li>
+                                            <li> The t-distribution </li>
+                                            <li> Factor analysis </li>
+                                            <li> The EM algorithm in detail </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial-5-variational-auto-encoders/" target="_blank"  rel="noreferrer">Variational autoencoders</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Non-linear latent variable models </li>
+                                            <li> Evidence lower bound (ELBO) </li>
+                                            <li> ELBO properties </li>
+                                            <li> Variational approximation </li>
+                                            <li> The variational autoencoder </li>
+                                            <li> Reparameterization trick </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://arxiv.org/abs/1908.09257" target="_blank"  rel="noreferrer">Normalizing flows: introduction and review</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                        <li> Normalizing flows </li>
+                                        <li> Elementwise and linear flows </li>
+                                        <li> Planar and radial flows </li>
+                                        <li> Coupling and auto-regressive flows </li>
+                                        <li> Coupling functions </li>
+                                        <li> Residual flows </li>
+                                        <li> Infinitesimal (continuous) flows </li>
+                                        <li> Datasets and performance </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                            </MoreOuterList>
+                            <TopLine>Graphical Models</TopLine>
+                            <MoreOuterList>
+                                <li>
+                                    <MoreLink href="https://drive.google.com/file/d/1ghgeRmeZMyzNHcuzVwS4vRP6BXi3npVO/view?usp=sharing" target="_blank"  rel="noreferrer">Graphical models</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Conditional independence </li>
+                                            <li> Directed graphical models </li>
+                                            <li> Undirected graphical models </li>
+                                            <li> Inference in graphical models </li>
+                                            <li> Sampling in graphical models </li>
+                                            <li> Learning in graphical models </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://drive.google.com/file/d/1WAMc3wtZoPv5wRkdF-D0SShVYF6Net84/view?usp=sharing" target="_blank"  rel="noreferrer">Models for chains and trees</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Hidden Markov models </li>
+                                            <li> Viterbi algorithm </li>
+                                            <li> Forward-backward algorithm </li>
+                                            <li> Belief propagation </li>
+                                            <li> Sum product algorithm </li>
+                                            <li> Extension to trees </li>
+                                            <li> Graphs with loops </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://drive.google.com/file/d/1qqS9OfA1z7t12M45UaBr4CSCj1jwzcwz/view?usp=sharing" target="_blank"  rel="noreferrer">Models for grids</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Markov random fields </li>
+                                            <li> MAP inference in binary pairwise MRFs </li>
+                                            <li> Graph cuts </li>
+                                            <li> Multi-label pairwise MRFs </li>
+                                            <li> Alpha-expansion algorithm </li>
+                                            <li> Conditional random fields </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                            </MoreOuterList>
+
+                            <TopLine>Machine learning</TopLine>
+                            <MoreOuterList>
+                                <li>
+                                    <MoreLink href="https://drive.google.com/file/d/1ArWWi-qbzK2ih6KpOeIF8wX5g3S4J5DY/view?usp=sharing" target="_blank"  rel="noreferrer">Learning and inference</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Discriminative models </li>
+                                            <li> Generative models </li>
+                                            <li> Example: regression </li>
+                                            <li> Example: classification </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://drive.google.com/file/d/1QZX5jm4xN8rhpvdjRsFP5Ybw1EXSNGaL/view?usp=sharing" target="_blank"  rel="noreferrer">Regression models</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Linear regression </li>
+                                            <li> Bayesian linear regression </li>
+                                            <li> Non-linear regression </li>
+                                            <li> Bayesian non-linear regression </li>
+                                            <li> The kernel trick </li>
+                                            <li> Gaussian process regression </li>
+                                            <li> Sparse linear regression </li>
+                                            <li> Relevance vector regression </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://drive.google.com/file/d/1-_f4Yfm8iBWcaZ2Gyjw6O0eZiODipmSV/view?usp=sharing" target="_blank"  rel="noreferrer">Classification models</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Logistic regression </li>
+                                            <li> Bayesian logistic regression </li>
+                                            <li> Non-linear logistic regression </li>
+                                            <li> Gaussian process classification </li>
+                                            <li> Relevance vector classification </li>
+                                            <li> Incremental fitting: boosting and trees </li>
+                                            <li> Multi-class logistic regression </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial-2-few-shot-learning-and-meta-learning-i/" target="_blank"  rel="noreferrer">Few-shot learning and meta-learning I</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Meta-learning framework </li>
+                                            <li> Approaches to meta-learning </li>
+                                            <li> Matching networks </li>
+                                            <li> Prototypical networks </li>
+                                            <li> Relation networks </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial-3-few-shot-learning-and-meta-learning-ii/" target="_blank"  rel="noreferrer">Few-shot learning and meta-learning II</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> MAML & Reptile </li>
+                                            <li> LSTM based meta-learning </li>
+                                            <li> Reinforcement learning based approaches</li>
+                                            <li> Memory augmented neural networks </li>
+                                            <li> SNAIL </li>
+                                            <li> Generative models </li>
+                                            <li> Data augmentation approaches </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                           </MoreOuterList>
+
+                           <TopLine>Natural language processing</TopLine>
+                           <MoreOuterList>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial-6-neural-natural-language-generation-decoding-algorithms/" target="_blank"  rel="noreferrer">Neural natural language generation I</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Encoder-decoder architecture </li>
+                                            <li> Maximum-likelihood training </li>
+                                            <li> Greedy search </li>
+                                            <li> Beam search </li>
+                                            <li> Diverse beam search </li>
+                                            <li> Top-k sampling </li>
+                                            <li> Nucleus sampling </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial-7-neural-natural-language-generation-sequence-level-training/" target="_blank"  rel="noreferrer">Neural natural language generation II</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Fine-tuning with reinforcement learning </li>
+                                            <li> Training from scratch with RL </li>
+                                            <li> RL vs. structured prediction </li>
+                                            <li> Minimum risk training </li>
+                                            <li> Scheduled sampling </li>
+                                            <li> Beam search optimization </li>
+                                            <li> SeaRNN </li>
+                                            <li> Reward-augmented maximum likelihood </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial-15-parsing-i-context-free-grammars-and-cyk-algorithm/" target="_blank"  rel="noreferrer">Parsing I</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Parse trees </li>
+                                            <li> Context-free grammars </li>
+                                            <li> Chomsky normal form </li>
+                                            <li> CYK recognition algorithm </li>
+                                            <li> Worked example </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial-18-parsing-ii-wcfgs-inside-algorithm-and-weighted-parsing/" target="_blank"  rel="noreferrer">Parsing II</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Weighted context-free grammars </li>
+                                            <li> Semirings </li>
+                                            <li> Inside algorithm </li>
+                                            <li> Inside weights </li>
+                                            <li> Weighted parsing </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial-19-parsing-iii-pcfgs-and-inside-outside-algorithm/" target="_blank"  rel="noreferrer">Parsing III</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Probabilistic context-free grammars </li>
+                                            <li> Parameter estimation (supervised) </li>
+                                            <li> Parameter estimation (unsupervised) </li>
+                                            <li> Viterbi training </li>
+                                            <li> Expectation maximization </li>
+                                            <li> Outside from inside </li>
+                                            <li> Interpretation of outside weights </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/understanding-xlnet/" target="_blank"  rel="noreferrer">XLNet</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Language modeling </li>
+                                            <li> XLNet training objective </li>
+                                            <li> Permutations </li>
+                                            <li> Attention mask </li>
+                                            <li> Two stream self-attention </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                            </MoreOuterList>
+
+       
+                           
+                           <TopLine>Responsible AI</TopLine>
+                           <MoreOuterList>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial1-bias-and-fairness-ai/" target="_blank"  rel="noreferrer">Bias and fairness</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Sources of bias</li>
+                                            <li> Demographic Parity </li>
+                                            <li> Equality of odds</li>
+                                            <li> Equality of opportunity </li>
+                                            <li> Individual fairness</li>
+                                            <li> Bias mitigation</li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/research-blogs/explainability-i-local-post-hoc-explanations/" target="_blank"  rel="noreferrer">Explainability I</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Taxonomy of XAI approaches</li>
+                                            <li> Local post-hoc explanations </li>
+                                            <li> Individual conditional explanation</li>
+                                            <li> Counterfactual explanations</li>
+                                            <li> LIME & Anchors</li>
+                                            <li> Shapley additive explanations & SHAP</li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/research-blogs/explainability-ii-global-explanations-proxy-models-and-interpretable-models/" target="_blank"  rel="noreferrer">Explainability II</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Global feature importance</li>
+                                            <li> Partial dependence & ICE plots</li>
+                                            <li> Accumulated local effects</li>
+                                            <li> Aggregate SHAP values</li>
+                                            <li> Prototypes & criticisms</li>
+                                            <li> Surrogate / proxy models</li>
+                                            <li> Inherently interpretable models</li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial-12-differential-privacy-i-introduction/" target="_blank"  rel="noreferrer">Differential privacy I</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Early approaches to privacy </li>
+                                            <li> Fundamental law of information recovery </li>
+                                            <li> Differential privacy</li>
+                                            <li> Properties of differential privacy </li>
+                                            <li> The Laplace mechanism</li>
+                                            <li> Examples</li>
+                                            <li> Other mechanisms and definitions</li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                                <li>
+                                    <MoreLink href="https://www.borealisai.com/en/blog/tutorial-13-differential-privacy-ii-machine-learning-and-data-generation/" target="_blank"  rel="noreferrer">Differential privacy II</MoreLink>
+                                    <MoreInnerP>
+                                        <MoreInnerList> 
+                                            <li> Differential privacy and matchine learning</li>
+                                            <li> DPSGD</li>
+                                            <li> PATE </li>
+                                            <li> Differentially private data generation</li>
+                                            <li> DPGAN</li>
+                                            <li> PateGAN </li>
+                                        </MoreInnerList>
+                                    </MoreInnerP>
+                                </li>
+                           </MoreOuterList>
+                        </Column2>
+                    </MoreRow2>
+                </MoreWrapper>
+            </MoreContainer>
+        </>
+    )
+}
+
+export default MoreSection
+
+

src/components/NavBar/NavbarElements.js

@@ -0,0 +1,119 @@
+import { Link as LinkS } from 'react-scroll';
+import { Link as LinkR } from 'react-router-dom';
+import styled from 'styled-components';
+
+export const Nav = styled.nav`
+  background: ${({ scrollNav }) => (scrollNav ? '#000' : 'transparent')};
+  height: 100px;
+  margin-top: -100px;
+  display: flex;
+  justify-content: center;
+  align-items: center;
+  font-size: 1rem;
+  position: sticky;
+  top: 0;
+  z-index: 10;
+
+  @media screen and (max-width: 960px) {
+    transition: 0.8s all ease;
+  }
+`;
+
+export const NavbarContainer = styled.div`
+  display: flex;
+  justify-content: space-between;
+  height: 100px;
+  z-index: 1;
+  width: 100%;
+  padding: 0 24px;
+  max-width: 1100px;
+`;
+
+export const NavLogo = styled(LinkR)`
+  color: #fff;
+  justify-self: flex-start;
+  cursor: pointer;
+  font-size: 1.5rem;
+  display: flex;
+  align-items: center;
+  margin-left: 24px;
+  font-weight: bold;
+  text-decoration: none;
+  @media screen and (max-width: 768px) {
+    font-size: 1.0rem;
+  }
+
+`;
+
+export const MobileIcon = styled.div`
+  display: none;
+
+  @media screen and (max-width: 768px) {
+    display: block;
+    position: absolute;
+    top: 0;
+    right: 0;
+    transform: translate(-100%, 60%);
+    font-size: 1.8rem;
+    cursor: pointer;
+  }
+`;
+
+export const NavMenu = styled.ul`
+  display: flex;
+  align-items: center;
+  list-style: none;
+  text-align: center;
+  margin-right: -22px;
+
+  @media screen and (max-width: 768px) {
+    display: none;
+  }
+`;
+
+export const NavItem = styled.li`
+  height: 80px;
+`;
+
+export const NavBtn = styled.nav`
+  display: flex;
+  align-items: center;
+
+  @media screen and (max-width: 768px) {
+    display: none;
+  }
+`;
+
+export const NavLinks = styled(LinkS)`
+  color: #fff;
+  display: flex;
+  align-items: center;
+  text-decoration: none;
+  padding: 0 1rem;
+  height: 100%;
+  cursor: pointer;
+
+  &.active {
+    border-bottom: 3px solid #57c6d1
+  }
+`;
+
+export const NavBtnLink = styled(LinkR)`
+  border-radius: 50px;
+  background: #01bf71;
+  white-space: nowrap;
+  padding: 10px 22px;
+  color: #010606;
+  font-size: 16px;
+  outline: none;
+  border: none;
+  cursor: pointer;
+  transition: all 0.2s ease-in-out;
+  text-decoration: none;
+
+  &:hover {
+    transition: all 0.2s ease-in-out;
+    background: #fff;
+    color: #010606;
+  }
+`;

src/components/NavBar/index.js

@@ -0,0 +1,59 @@
+import React, {useState, useEffect} from 'react'
+import {FaBars} from 'react-icons/fa'
+import {IconContext} from 'react-icons/lib'
+import {Nav, NavbarContainer, NavLogo, MobileIcon, NavMenu, NavItem, NavLinks} from './NavbarElements'
+import { animateScroll as scroll } from 'react-scroll'
+
+
+const Navbar = ( {toggle} ) => {
+    const [scrollNav, setScrollNav] = useState(false)
+
+    const changeNav = () =>{
+        if (window.scrollY >= 80){
+            setScrollNav(true)
+        }else{
+            setScrollNav(false)
+        }
+    }
+
+    useEffect(() =>{
+        window.addEventListener('scroll', changeNav)
+    }, [])
+
+    const toggleHome = () => {
+        scroll.scrollToTop();
+    }
+
+    return (
+        <>
+        <IconContext.Provider value={{color: '#fff'}}>
+            <Nav scrollNav={scrollNav}>
+                <NavbarContainer>
+                    <NavLogo to="/udlbook/" onClick={toggleHome}>
+                        <h1> Understanding Deep Learning </h1>
+                    </NavLogo>
+                    <MobileIcon onClick={toggle}>
+                        <FaBars />
+                    </MobileIcon>
+                    <NavMenu>
+                        <NavItem>
+                            <NavLinks to="Notebooks" smooth={true} duration={500} spy={true} exact='true' offset={-80} activeClass='active'>Notebooks</NavLinks>
+                        </NavItem>
+                        <NavItem>
+                            <NavLinks to="Instructors" smooth={true} duration={500} spy={true} exact='true' offset={-80} activeClass='active'>Instructors</NavLinks>
+                        </NavItem>
+                        <NavItem>
+                            <NavLinks to="Media" smooth={true} duration={500} spy={true} exact='true' offset={-80} activeClass='active'>Media</NavLinks>
+                        </NavItem>
+                        <NavItem>
+                            <NavLinks to="More" smooth={true} duration={500} spy={true} exact='true' offset={-80} activeClass='active'>More</NavLinks>
+                        </NavItem>
+                    </NavMenu>
+                </NavbarContainer>
+            </Nav>
+        </IconContext.Provider>
+        </>
+    );
+}
+
+export default Navbar

src/components/Notebooks/NotebookElements.js

@@ -0,0 +1,147 @@
+import styled from "styled-components";
+
+
+export const NotebookContainer = styled.div` 
+    color: #fff;
+    /* background: #f9f9f9; */
+    background: ${({lightBg}) => (lightBg ? '#f9f9f9': '#010606')};
+
+    @media screen and (max-width: 768px) {
+        padding: 100px 0;
+    }
+`
+
+export const NotebookWrapper = styled.div`
+    display: grid ; 
+    z-index: 1;
+    // height: 1250px ;  
+    width: 100% ; 
+    max-width: 1100px;
+    margin-right: auto;
+    margin-left: auto;
+    padding: 0 24px;
+    justify-content: center;
+`
+
+export const NotebookRow = styled.div` 
+    display: grid; 
+    grid-auto-columns: minmax(auto, 1fr);
+    align-items: center;
+    grid-template-areas: ${({imgStart}) => (imgStart ? `'col2 col1'` : `'col1 col2'`)};
+
+    @media screen and (max-width: 768px){
+        grid-template-areas: ${({imgStart}) => (imgStart ? `'col1' 'col2'` : `'col1 col1' 'col2 col2'`)};
+    }
+`
+
+export const Column1 = styled.p` 
+    margin-bottom: 15px;
+    padding: 0 15px;
+    grid-area: col1;
+    @media screen and (max-width: 1050px) {
+        font-size: 12px;
+    }
+
+    @media screen and (max-width: 768px) {
+        font-size: 10px;
+    }
+`
+
+export const Column2 = styled.p` 
+    margin-bottom: 15px;
+    padding: 0 15px;
+    grid-area: col2;
+    @media screen and (max-width: 1050px) {
+        font-size: 12px;
+    }
+
+    @media screen and (max-width: 768px) {
+        font-size: 10px;
+    }
+`
+
+export const TextWrapper = styled.div`
+    max-width: 540px ; 
+    padding-top: 0;
+    padding-bottom: 0;
+`
+
+export const TopLine = styled.p` 
+    color: #57c6d1;
+    font-size: 16px;
+    line-height: 16px;
+    font-weight: 700;
+    letter-spacing: 1.4px;
+    text-transform: uppercase;
+    margin-bottom: 16px;
+`
+
+export const Heading= styled.h1` 
+
+    margin-bottom: 24px;
+    font-size: 48px;
+    line-height: 1.1;
+    font-weight: 600;
+    color: ${({lightText}) => (lightText ? '#f7f8fa' : '#010606')};
+    
+    @media screen and (max-width: 480px)
+    {
+        font-size: 32px;
+    }
+`
+
+
+export const Subtitle = styled.p` 
+    max-width: 440px;
+    margin-bottom: 35px;
+    font-size: 18px;
+    line-height: 24px;
+    color: ${({darkText})=> (darkText ? '#010606' : '#fff')};
+`
+
+export const BtnWrap = styled.div`
+    display: flex;
+    justify-content: flex-start;
+`
+
+export const ImgWrap = styled.div` 
+    max-width: 555px;
+    height: 100%;
+`
+
+export const Img = styled.img`
+  width: 100%;
+  margin-top: 0;
+  margin-right: 0;
+  margin-left: 10px;
+  padding-right: 0;
+`;
+
+
+export const NBLink = styled.a`
+    text-decoration: none;
+    color: #57c6d1;;
+    font-weight: 300;
+    margin: 0 2px;
+    position: relative;
+
+    &:before{
+        position: absolute;
+        margin: 0 auto;
+        top: 100%;
+        left: 0;
+        width: 100%;
+        height: 2px;
+        background-color: #57c6d1;;
+        content: '';
+        opacity: .3;
+        -webkit-transform: scaleX(1);
+        transition-property: opacity, -webkit-transform;
+        transition-duration: .3s;
+    }
+
+    &:hover:before {
+        opacity: 1;
+        -webkit-transform: scaleX(1.05);
+      }
+`

src/components/Notebooks/index.js

@@ -0,0 +1,220 @@
+import React from 'react'
+import { NBLink, ImgWrap, Img, NotebookContainer, NotebookWrapper, NotebookRow, Column1, Column2, TextWrapper, TopLine, Heading, Subtitle} from './NotebookElements'
+
+// export const homeObjOne = {
+//     id: 'about',
+//     lightBg: false,
+//     lightText: true,
+//     lightTextDesc: true,
+//     topLine: 'Premium Bank',
+//     headline: 'Unlimited transactions with zero fees',
+//     description:
+//       'Get access to our exclusive app that allows you to send unlimited transactions without getting charged any fees',
+//     buttonLabel: 'Get Started',
+//     imgStart: false,
+//     img: require('../../images/svg-1.svg').default,
+//     alt: 'Car',
+//     dark: true,
+//     primary: true,
+//     darkText: false
+//   };
+
+import img from '../../images/coding.svg'
+
+
+
+const NotebookSection = () => {
+    return (
+        <>
+            <NotebookContainer lightBg={false} id='Notebooks'>
+                <NotebookWrapper>
+                    <NotebookRow imgStart={true}>
+                        <Column1>
+                            <TextWrapper>
+                                <TopLine>Coding exercises</TopLine>
+                                <Heading lightText={true}>Python notebooks covering the whole text</Heading>
+                                <Subtitle darkText={false}>Sixty eight python notebook exercises with missing code to fill in based on the text</Subtitle>
+                            </TextWrapper>
+                        </Column1>
+                        <Column2>
+                            <ImgWrap>
+                                <Img src={img} alt='Car'/>
+                            </ImgWrap>
+                        </Column2>
+                    </NotebookRow>
+                    <NotebookRow>
+                        <Column1>
+                            <ul>
+                                <li> Notebook 1.1 - Background mathematics: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap01/1_1_BackgroundMathematics.ipynb">ipynb/colab</NBLink>
+                                </li>
+                                <li> Notebook 2.1 - Supervised learning: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap02/2_1_Supervised_Learning.ipynb">ipynb/colab</NBLink>
+                                </li>
+                                <li> Notebook 3.1 - Shallow networks I: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_1_Shallow_Networks_I.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 3.2 - Shallow networks II: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_2_Shallow_Networks_II.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 3.3 - Shallow network regions: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_3_Shallow_Network_Regions.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 3.4 - Activation functions: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_4_Activation_Functions.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 4.1 - Composing networks: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap04/4_1_Composing_Networks.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 4.2 - Clipping functions: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap04/4_2_Clipping_functions.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 4.3 - Deep networks: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap04/4_3_Deep_Networks.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 5.1 - Least squares loss: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap05/5_1_Least_Squares_Loss.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 5.2 - Binary cross-entropy loss: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap05/5_2_Binary_Cross_Entropy_Loss.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 5.3 - Multiclass cross-entropy loss: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap05/5_3_Multiclass_Cross_entropy_Loss.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 6.1 - Line search: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_1_Line_Search.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 6.2 - Gradient descent: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_2_Gradient_Descent.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 6.3 - Stochastic gradient descent: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_3_Stochastic_Gradient_Descent.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 6.4 - Momentum: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_4_Momentum.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 6.5 - Adam: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_5_Adam.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 7.1 - Backpropagation in toy model: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap07/7_1_Backpropagation_in_Toy_Model.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 7.2 - Backpropagation: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap07/7_2_Backpropagation.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 7.3 - Initialization: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap07/7_3_Initialization.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 8.1 - MNIST-1D performance: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap08/8_1_MNIST_1D_Performance.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 8.2 - Bias-variance trade-off: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap08/8_2_Bias_Variance_Trade_Off.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 8.3 - Double descent: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap08/8_3_Double_Descent.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 8.4 - High-dimensional spaces: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap08/8_4_High_Dimensional_Spaces.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 9.1 - L2 regularization: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap09/9_1_L2_Regularization.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 9.2 - Implicit regularization: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap09/9_2_Implicit_Regularization.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 9.3 - Ensembling: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap09/9_3_Ensembling.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 9.4 - Bayesian approach: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap09/9_4_Bayesian_Approach.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 9.5 - Augmentation <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap09/9_5_Augmentation.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 10.1 - 1D convolution: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap10/10_1_1D_Convolution.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 10.2 - Convolution for MNIST-1D: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap10/10_2_Convolution_for_MNIST_1D.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 10.3 - 2D convolution: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap10/10_3_2D_Convolution.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 10.4 - Downsampling & upsampling: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap10/10_4_Downsampling_and_Upsampling.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 10.5 - Convolution for MNIST: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap10/10_5_Convolution_For_MNIST.ipynb">ipynb/colab </NBLink>
+                                </li>
+                            </ul>
+                        </Column1>
+                        <Column2>
+                        <ul>
+                                <li> Notebook 11.1 - Shattered gradients: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap11/11_1_Shattered_Gradients.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 11.2 - Residual networks: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap11/11_2_Residual_Networks.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 11.3 - Batch normalization: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap11/11_3_Batch_Normalization.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 12.1 - Self-attention: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap12/12_1_Self_Attention.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 12.2 - Multi-head self-attention: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap12/12_2_Multihead_Self_Attention.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 12.3 - Tokenization: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap12/12_3_Tokenization.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 12.4 - Decoding strategies: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap12/12_4_Decoding_Strategies.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 13.1 - Encoding graphs: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap13/13_1_Graph_Representation.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 13.2 - Graph classification : <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap13/13_2_Graph_Classification.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 13.3 - Neighborhood sampling: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap13/13_3_Neighborhood_Sampling.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 13.4 - Graph attention: <NBLink
+                                        href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap13/13_4_Graph_Attention_Networks.ipynb">ipynb/colab </NBLink>
+                                </li>
+                                <li> Notebook 15.1 - GAN toy example: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap15/15_1_GAN_Toy_Example.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 15.2 - Wasserstein distance: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap15/15_2_Wasserstein_Distance.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 16.1 - 1D normalizing flows: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap16/16_1_1D_Normalizing_Flows.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 16.2 - Autoregressive flows: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap16/16_2_Autoregressive_Flows.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 16.3 - Contraction mappings: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap16/16_3_Contraction_Mappings.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 17.1 - Latent variable models: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap17/17_1_Latent_Variable_Models.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 17.2 - Reparameterization trick: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap17/17_2_Reparameterization_Trick.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 17.3 - Importance sampling: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap17/17_3_Importance_Sampling.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 18.1 - Diffusion encoder: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap18/18_1_Diffusion_Encoder.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 18.2 - 1D diffusion model: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap18/18_2_1D_Diffusion_Model.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 18.3 - Reparameterized model: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap18/18_3_Reparameterized_Model.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 18.4 - Families of diffusion models: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap18/18_4_Families_of_Diffusion_Models.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 19.1 - Markov decision processes: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap19/19_1_Markov_Decision_Processes.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 19.2 - Dynamic programming: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap19/19_2_Dynamic_Programming.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 19.3 - Monte-Carlo methods: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap19/19_3_Monte_Carlo_Methods.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 19.4 - Temporal difference methods: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap19/19_4_Temporal_Difference_Methods.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 19.5 - Control variates: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap19/19_5_Control_Variates.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 20.1 - Random data: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap20/20_1_Random_Data.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 20.2 - Full-batch gradient descent: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap20/20_2_Full_Batch_Gradient_Descent.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 20.3 - Lottery tickets: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap20/20_3_Lottery_Tickets.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 20.4 - Adversarial attacks: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap20/20_4_Adversarial_Attacks.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 21.1 - Bias mitigation: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap21/21_1_Bias_Mitigation.ipynb">ipynb/colab </NBLink></li>
+                                <li> Notebook 21.2 - Explainability: <NBLink href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap21/21_2_Explainability.ipynb">ipynb/colab </NBLink></li>
+                            </ul>
+                        </Column2>
+                    </NotebookRow>
+
+                </NotebookWrapper>
+            </NotebookContainer>
+        </>
+    )
+}
+
+export default NotebookSection

src/components/ScrollToTop.js

@@ -0,0 +1,11 @@
+import {useEffect} from 'react'
+import { useLocation } from 'react-router-dom'
+
+export default function ScrollToTop() {
+    const {pathname} = useLocation()
+    useEffect(() => {
+        window.scrollTo(0,0)
+    }, [pathname])
+
+    return null;
+}

src/components/Sidebar/SidebarElements.js

@@ -0,0 +1,98 @@
+import styled from 'styled-components'
+import {Link as LinkS} from 'react-scroll'
+import {Link as LinkR} from 'react-router-dom'
+import {FaTimes} from 'react-icons/fa'
+
+
+
+export const SidebarContainer = styled.aside`
+    position:fixed ; 
+    z-index: 999;
+    width: 100%;
+    height: 100%;
+    background: #0d0d0d;
+    display: grid;
+    align-items: center;
+    top: 0;
+    left: 0;
+    transition: 0.3s ease-in-out;
+    opacity: ${({ isOpen }) => (isOpen ? '100%' :  '0')};
+    top: ${({ isOpen }) => (isOpen ? '0' : '-100%')};   
+ 
+`
+
+export const CloseIcon = styled(FaTimes)`
+    color: #fff ; 
+    &:hover {
+        color: #01bf71;
+        transition: 0.2s ease-in-out;
+    }
+`
+
+export const Icon = styled.div`
+    position: absolute;
+    top: 1.2rem;
+    right: 1.5rem;
+    background: transparent;
+    font-size: 2rem;
+    cursor: pointer;
+    outline: none;
+`
+
+export const SidebarWrapper = styled.div`
+    color: #ffffff;
+`
+
+export const SidebarMenu = styled.ul`
+    display: grid;
+    grid-template-columns: 1fr;
+    grid-template-rows: repeat(6,80px);
+    text-align: center;
+
+    @media screen and (max-width: 480px){
+        grid-template-rows: repeat(6, 60px) ; 
+    }
+`
+
+export const SidebarLink = styled(LinkS)`
+    display: flex ;
+    align-items: center;
+    justify-content: center;
+    font-size: 1.5rem;
+    text-decoration: none;
+    list-style: none;
+    transition: 0.2s ease-in-out;
+    text-decoration: none;
+    color: #fff;
+    cursor: pointer;
+
+    &:hover {
+        color: #01bf71;
+        transition: 0.2s ease-in-out;
+    }
+`
+
+export const SideBtnWrap = styled.div` 
+    display: flex;
+    justify-content: center;
+`
+
+export const SidebarRoute = styled(LinkR)` 
+    border-radius: 50px;
+    background: #01bf71;
+    white-space: nowrap;
+    padding: 16px 46px;
+    color: #010606;
+    font-size: 16px;
+    outline: none;
+    border: none;
+    cursor: pointer;
+    transition: all 0.2s ease-in-out;
+    text-decoration: none;
+
+    &:hover {
+        transition: all 0.2s ease-in-out;
+        background: #fff;
+        color: #010606;
+    }
+`

src/components/Sidebar/index.js

@@ -0,0 +1,33 @@
+import React from 'react'
+import { SidebarContainer, Icon, CloseIcon, SidebarWrapper, SidebarMenu, SidebarLink} from './SidebarElements'
+
+
+const Sidebar = ({isOpen, toggle}) => {
+    return (
+        <>
+        <SidebarContainer isOpen={isOpen} onClick={toggle}>
+            <Icon onClick={toggle}>
+                <CloseIcon />
+            </Icon>
+            <SidebarWrapper>
+                <SidebarMenu >
+                    <SidebarLink to="Notebooks" onClick={toggle}>
+                        Notebooks
+                    </SidebarLink>
+                    <SidebarLink to="Instructors" onClick={toggle}>
+                        Instructors
+                    </SidebarLink>
+                    <SidebarLink to="Media" onClick={toggle}>
+                        Media
+                    </SidebarLink>
+                    <SidebarLink to="More" onClick={toggle}>
+                        More
+                    </SidebarLink>
+                </SidebarMenu>
+            </SidebarWrapper>
+        </SidebarContainer>
+        </>
+    )
+}
+
+export default Sidebar

src/images/square-x-twitter.svg

@@ -0,0 +1,39 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<svg
+   height="10"
+   width="8.75"
+   viewBox="0 0 448 512"
+   version="1.1"
+   id="svg914"
+   sodipodi:docname="square-x-twitter.svg"
+   inkscape:version="1.1.2 (b8e25be8, 2022-02-05)"
+   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+   xmlns="http://www.w3.org/2000/svg"
+   xmlns:svg="http://www.w3.org/2000/svg">
+  <defs
+     id="defs918" />
+  <sodipodi:namedview
+     id="namedview916"
+     pagecolor="#ffffff"
+     bordercolor="#666666"
+     borderopacity="1.0"
+     inkscape:pageshadow="2"
+     inkscape:pageopacity="0.0"
+     inkscape:pagecheckerboard="0"
+     showgrid="false"
+     inkscape:zoom="65.6"
+     inkscape:cx="3.8948171"
+     inkscape:cy="4.5426829"
+     inkscape:window-width="1296"
+     inkscape:window-height="906"
+     inkscape:window-x="0"
+     inkscape:window-y="38"
+     inkscape:window-maximized="0"
+     inkscape:current-layer="svg914" />
+  <!--!Font Awesome Free 6.5.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free Copyright 2024 Fonticons, Inc.-->
+  <path
+     d="M64 32C28.7 32 0 60.7 0 96V416c0 35.3 28.7 64 64 64H384c35.3 0 64-28.7 64-64V96c0-35.3-28.7-64-64-64H64zm297.1 84L257.3 234.6 379.4 396H283.8L209 298.1 123.3 396H75.8l111-126.9L69.7 116h98l67.7 89.5L313.6 116h47.5zM323.3 367.6L153.4 142.9H125.1L296.9 367.6h26.3z"
+     id="path912"
+     style="fill:#ffffff;fill-opacity:1" />
+</svg>

src/index.js

@@ -0,0 +1,11 @@
+import React from 'react';
+import ReactDOM from 'react-dom';
+import App from './App';
+
+ReactDOM.render(
+  <React.StrictMode>
+    <App />
+  </React.StrictMode>,
+  document.getElementById('root')
+);
+

src/pages/index.js

@@ -0,0 +1,34 @@
+import React, {useState} from 'react'
+import Sidebar from '../components/Sidebar'
+import Navbar from '../components/NavBar'
+import HeroSection from '../components/HeroSection';
+import NotebookSection from '../components/Notebooks'
+import InstructorsSection from '../components/Instructors';
+import Footer from '../components/Footer';
+import MediaSection from '../components/Media';
+import MoreSection from '../components/More';
+
+const Home = () => {
+    const [isOpen, setIsOpen] = useState(false)
+
+    const toggle = () => {
+        setIsOpen(!isOpen)
+    };
+
+    return ( 
+        <>
+            <Sidebar isOpen={isOpen} toggle={toggle}/>
+            <Navbar toggle={toggle}/>
+            <HeroSection />
+            <NotebookSection/>
+            <InstructorsSection/>
+            <MediaSection/>
+            <MoreSection/>
+            <Footer/>                
+        </>
+    )
+};
+
+export default Home
+
+ 

src/pages/signin.js

@@ -0,0 +1,14 @@
+import React from 'react'
+import ScrollToTop from '../components/ScrollToTop';
+import SignIn from '../components/SignIn';
+
+const SigninPage = () => {
+    return (
+        <>
+            <ScrollToTop />
+            <SignIn />
+        </>
+    )
+}
+
+export default SigninPage;

style.css

@@ -1,23 +0,0 @@
-body {
-    font-size: 17px;
-    margin: 2% 10%;
-}
-
-#head {
-    display: flex;
-    flex-direction: row;
-    flex-wrap: wrap-reverse;
-    justify-content: space-between;
-    width: 100%;
-}
-
-#cover {
-    justify-content: center;
-    display: flex;
-    width: 30%;
-}
-
-#cover img {
-    width: 100%;
-    height: min-content;
-}