dr_domi/udlbook
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'main' of https://github.com/udlbook/udlbook

Browse Source 
commit.
This commit is contained in:
Simon Prince
2025-03-28 14:45:38 -04:00
parent 2016977f30 f88127c0d2
commit 884a7e358b
82 changed files with 3176 additions and 169 deletions
Download Patch File Download Diff File Expand all files Collapse all files

432
Blogs/BorealisODENumerical.ipynb Normal file
View File

@@ -0,0 +1,432 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/Blogs/BorealisODENumerical.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JXsO7ce7oqeq"
},
"source": [
"# Numerical methods for ODEs\n",
"\n",
"This blog contains code that accompanies the RBC Borealis blog on numerical methods for ODEs. Contact udlbookmail@gmail.com if you find any problems."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "AnvAKtP_oqes"
},
"source": [
"Import relevant libraries"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "UF-gJyZggyrl"
},
"outputs": [],
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "szWLVrSSoqet"
},
"source": [
"Define the ODE that we will be experimenting with."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "NkrGZLL6iM3P"
},
"outputs": [],
"source": [
"# The ODE that we will experiment with\n",
"def ode_lin_homog(t,x):\n",
" return 0.5 * x ;\n",
"\n",
"# The derivative of the ODE function with respect to x (needed for Taylor's method)\n",
"def ode_lin_homog_deriv_x(t,x):\n",
" return 0.5 ;\n",
"\n",
"# The derivative of the ODE function with respect to t (needed for Taylor's method)\n",
"def ode_lin_homog_deriv_t(t,x):\n",
" return 0.0 ;\n",
"\n",
"# The closed form solution (so we can measure the error)\n",
"def ode_lin_homog_soln(t,C=0.5):\n",
" return C * np.exp(0.5 * t) ;"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "In1C9wZkoqet"
},
"source": [
"This is a generic method that runs the numerical methods. It takes the initial conditions ($t_0$, $x_0$), the final time $t_1$ and the step size $h$. It also takes the ODE function itself and its derivatives (only used for Taylor's method). Finally, the parameter \"step_function\" is the method used to update (e.g., Euler's methods, Runge-Kutte 4-step)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "VZfZDJAfmyrf"
},
"outputs": [],
"source": [
"def run_numerical(x_0, t_0, t_1, h, ode_func, ode_func_deriv_x, ode_func_deriv_t, ode_soln, step_function):\n",
" x = [x_0]\n",
" t = [t_0]\n",
" while (t[-1] <= t_1):\n",
" x = x+[step_function(x[-1],t[-1],h, ode_func, ode_func_deriv_x, ode_func_deriv_t)]\n",
" t = t + [t[-1]+h]\n",
"\n",
" # Returns x,y plot plus total numerical error at last point.\n",
" return t, x, np.abs(ode_soln(t[-1])-x[-1])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Vfkc3-_7oqet"
},
"source": [
"Run the numerical method with step sizes of 2.0, 1.0, 0.5, 0.25, 0.125, 0.0675 and plot the results"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1tyGbMZhoqeu"
},
"outputs": [],
"source": [
"def run_and_plot(ode, ode_deriv_x, ode_deriv_t, ode_solution, step_function):\n",
" # Specify the grid of points to draw the ODE\n",
" t = np.arange(0.04, 4.0, 0.2)\n",
" x = np.arange(0.04, 4.0, 0.2)\n",
" T, X = np.meshgrid(t,x)\n",
"\n",
" # ODE equation at these grid points (used to draw quiver-plot)\n",
" dx = ode(T,X)\n",
" dt = np.ones(dx.shape)\n",
"\n",
" # The ground truth solution\n",
" t2= np.arange(0,10,0.1)\n",
" x2 = ode_solution(t2)\n",
"\n",
" #####################################x_0, t_0, t_1, h #################################################\n",
" t_sim1,x_sim1,error1 = run_numerical(0.5, 0.0, 4.0, 2.0000, ode, ode_deriv_x, ode_deriv_t, ode_solution, step_function)\n",
" t_sim2,x_sim2,error2 = run_numerical(0.5, 0.0, 4.0, 1.0000, ode, ode_deriv_x, ode_deriv_t, ode_solution, step_function)\n",
" t_sim3,x_sim3,error3 = run_numerical(0.5, 0.0, 4.0, 0.5000, ode, ode_deriv_x, ode_deriv_t, ode_solution, step_function)\n",
" t_sim4,x_sim4,error4 = run_numerical(0.5, 0.0, 4.0, 0.2500, ode, ode_deriv_x, ode_deriv_t, ode_solution, step_function)\n",
" t_sim5,x_sim5,error5 = run_numerical(0.5, 0.0, 4.0, 0.1250, ode, ode_deriv_x, ode_deriv_t, ode_solution, step_function)\n",
" t_sim6,x_sim6,error6 = run_numerical(0.5, 0.0, 4.0, 0.0675, ode, ode_deriv_x, ode_deriv_t, ode_solution, step_function)\n",
"\n",
" # Plot the ODE and ground truth solution\n",
" fig,ax = plt.subplots()\n",
" ax.quiver(T,X,dt,dx, scale=35.0)\n",
" ax.plot(t2,x2,'r-')\n",
"\n",
" # Plot the numerical approximations\n",
" ax.plot(t_sim1,x_sim1,'.-',markeredgecolor='#773c23ff',markerfacecolor='#d18362', color='#d18362', markersize=10)\n",
" ax.plot(t_sim2,x_sim2,'.-',markeredgecolor='#773c23ff',markerfacecolor='#d18362', color='#d18362', markersize=10)\n",
" ax.plot(t_sim3,x_sim3,'.-',markeredgecolor='#773c23ff',markerfacecolor='#d18362', color='#d18362', markersize=10)\n",
" ax.plot(t_sim4,x_sim4,'.-',markeredgecolor='#773c23ff',markerfacecolor='#d18362', color='#d18362', markersize=10)\n",
" ax.plot(t_sim5,x_sim5,'.-',markeredgecolor='#773c23ff',markerfacecolor='#d18362', color='#d18362', markersize=10)\n",
" ax.plot(t_sim6,x_sim6,'.-',markeredgecolor='#773c23ff',markerfacecolor='#d18362', color='#d18362', markersize=10)\n",
"\n",
" ax.set_aspect('equal')\n",
" ax.set_xlim(0,4)\n",
" ax.set_ylim(0,4)\n",
"\n",
" plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JYrq8QIwvOIy"
},
"source": [
"# Euler Method\n",
"\n",
"Define the Euler method and set up functions for plotting."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "N73xMnCukVVX"
},
"outputs": [],
"source": [
"def euler_step(x_0, t_0, h, ode_func, ode_func_deriv_x=None, ode_func_deriv_t=None):\n",
" return x_0 + h * ode_func(t_0, x_0) ;"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "4B1_PGEcsZ9H"
},
"outputs": [],
"source": [
"run_and_plot(ode_lin_homog, None, None, ode_lin_homog_soln, euler_step)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FfwNihtkvJeX"
},
"source": [
"# Heun's Method"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "srHfNDcDxI1o"
},
"outputs": [],
"source": [
"def heun_step(x_0, t_0, h, ode_func, ode_func_deriv_x=None, ode_func_deriv_t=None):\n",
" f_x0_t0 = ode_func(t_0, x_0)\n",
" return x_0 + h/2 * ( f_x0_t0 + ode_func(t_0+h, x_0+h*f_x0_t0)) ;"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "WOApHz9xoqev"
},
"outputs": [],
"source": [
"run_and_plot(ode_lin_homog, None, None, ode_lin_homog_soln, heun_step)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0XSzzFDIvRhm"
},
"source": [
"# Modified Euler method"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "fSXprgVJ5Yep"
},
"outputs": [],
"source": [
"def modified_euler_step(x_0, t_0, h, ode_func, ode_func_deriv_x=None, ode_func_deriv_t=None):\n",
" f_x0_t0 = ode_func(t_0, x_0)\n",
" return x_0 + h * ode_func(t_0+h/2, x_0+ h * f_x0_t0/2) ;"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "8LKSrCD2oqev"
},
"outputs": [],
"source": [
"run_and_plot(ode_lin_homog, None, None, ode_lin_homog_soln, modified_euler_step)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "yp8ZBpwooqev"
},
"source": [
"# Second order Taylor's method"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "NtBBgzWLoqev"
},
"outputs": [],
"source": [
"def taylor_2nd_order(x_0, t_0, h, ode_func, ode_func_deriv_x, ode_func_deriv_t):\n",
" f1 = ode_func(t_0, x_0)\n",
" return x_0 + h * ode_func(t_0, x_0) + (h*h/2) * (ode_func_deriv_x(t_0,x_0) * ode_func(t_0, x_0) + ode_func_deriv_t(t_0, x_0))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ioeeIohUoqev"
},
"outputs": [],
"source": [
"run_and_plot(ode_lin_homog, ode_lin_homog_deriv_x, ode_lin_homog_deriv_t, ode_lin_homog_soln, taylor_2nd_order)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WcuhV5lL1zAJ"
},
"source": [
"# Fourth Order Runge Kutta"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0NZN81Bpwu56"
},
"outputs": [],
"source": [
"def runge_kutta_4_step(x_0, t_0, h, ode_func, ode_func_deriv_x=None, ode_func_deriv_t=None):\n",
" f1 = ode_func(t_0, x_0)\n",
" f2 = ode_func(t_0+h/2,x_0+f1 * h/2)\n",
" f3 = ode_func(t_0+h/2,x_0+f2 * h/2)\n",
" f4 = ode_func(t_0+h, x_0+ f3*h)\n",
" return x_0 + (h/6) * (f1 + 2*f2 + 2*f3+f4)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "K-OxE9E6oqew"
},
"outputs": [],
"source": [
"run_and_plot(ode_lin_homog, None, None, ode_lin_homog_soln, runge_kutta_4_step)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "7JifxBhhoqew"
},
"source": [
"# Plot the error as a function of step size"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ZoEpmlCfsi9P"
},
"outputs": [],
"source": [
"# Run systematically with a number of different step sizes and store errors for each\n",
"def get_errors(ode, ode_deriv_x, ode_deriv_t, ode_solution, step_function):\n",
" # Choose the step size h to divide the plotting interval into 1,2,4,8... segments.\n",
" # The plots in the article add a few more smaller step sizes, but this takes a while to compute.\n",
" # Add them back in if you want the full plot.\n",
" all_h = (1./np.array([1,2,4,8,16,32,64,128,256,512,1024,2048,4096])).tolist()\n",
" all_err = []\n",
"\n",
" for i in range(len(all_h)):\n",
" t_sim,x_sim,err = run_numerical(0.5, 0.0, 4.0, all_h[i], ode, ode_deriv_x, ode_deriv_t, ode_solution, step_function)\n",
" all_err = all_err + [err]\n",
"\n",
" return all_h, all_err"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "X0O0KK47xF28"
},
"outputs": [],
"source": [
"# Plot the errors\n",
"all_h, all_err_euler = get_errors(ode_lin_homog, ode_lin_homog_deriv_x, ode_lin_homog_deriv_t, ode_lin_homog_soln, euler_step)\n",
"all_h, all_err_heun = get_errors(ode_lin_homog, ode_lin_homog_deriv_x, ode_lin_homog_deriv_t, ode_lin_homog_soln, heun_step)\n",
"all_h, all_err_mod_euler = get_errors(ode_lin_homog, ode_lin_homog_deriv_x, ode_lin_homog_deriv_t, ode_lin_homog_soln, modified_euler_step)\n",
"all_h, all_err_taylor = get_errors(ode_lin_homog, ode_lin_homog_deriv_x, ode_lin_homog_deriv_t, ode_lin_homog_soln, taylor_2nd_order)\n",
"all_h, all_err_rk = get_errors(ode_lin_homog, ode_lin_homog_deriv_x, ode_lin_homog_deriv_t, ode_lin_homog_soln, runge_kutta_4_step)\n",
"\n",
"\n",
"fig, ax = plt.subplots()\n",
"ax.loglog(all_h, all_err_euler,'ro-')\n",
"ax.loglog(all_h, all_err_heun,'bo-')\n",
"ax.loglog(all_h, all_err_mod_euler,'go-')\n",
"ax.loglog(all_h, all_err_taylor,'co-')\n",
"ax.loglog(all_h, all_err_rk,'mo-')\n",
"ax.set_ylim(1e-13,1e1)\n",
"ax.set_xlim(1e-6,1e1)\n",
"ax.set_aspect(0.5)\n",
"ax.set_xlabel('Step size, $h$')\n",
"ax.set_ylabel('Error')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "BttOqpeo9MsJ"
},
"source": [
"Note that for this ODE, the Heun, Modified Euler and Taylor methods provide EXACTLY the same updates, and so the error curves for all three are identical (subject to difference is numerical rounding errors). This is not in general the case, although the general trend would be the same for each."
]
}
],
"metadata": {
"colab": {
"provenance": [],
"include_colab_link": true
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.10"
}
},
"nbformat": 4,
"nbformat_minor": 0
}

2
Notebooks/Chap01/1_1_BackgroundMathematics.ipynb
View File

@@ -19,7 +19,7 @@
"\n",
"# **Notebook 1.1 -- Background Mathematics**\n",
"\n",
"The purpose of this Python notebook is to make sure you can use CoLab and to familiarize yourself with some of the background mathematical concepts that you are going to need to understand deep learning. <br><br> It's not meant to be difficult and it may be that you know some or all of this information already.<br><br> Math is *NOT* a spectator sport. You won't learn it by just listening to lectures or reading books. It really helps to interact with it and explore yourself. <br><br> Work through the cells below, running each cell in turn. In various places you will see the words **\"TO DO\"**. Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
"The purpose of this Python notebook is to make sure you can use CoLab and to familiarize yourself with some of the background mathematical concepts that you are going to need to understand deep learning. <br><br> It's not meant to be difficult and it may be that you know some or all of this information already.<br><br> Math is *NOT* a spectator sport. You won't learn it by just listening to lectures or reading books. It really helps to interact with it and explore yourself. <br><br> Work through the cells below, running each cell in turn. In various places you will see the words **\"TODO\"**. Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
]

2
Notebooks/Chap02/2_1_Supervised_Learning.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"The purpose of this notebook is to explore the linear regression model discussed in Chapter 2 of the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

2
Notebooks/Chap03/3_1_Shallow_Networks_I.ipynb
View File

@@ -20,7 +20,7 @@
"\n",
"The purpose of this notebook is to gain some familiarity with shallow neural networks with 1D inputs. It works through an example similar to figure 3.3 and experiments with different activation functions. <br>\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
]

2
Notebooks/Chap03/3_2_Shallow_Networks_II.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"The purpose of this notebook is to gain some familiarity with shallow neural networks with 2D inputs. It works through an example similar to figure 3.8 and experiments with different activation functions. <br><br>\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
],

2
Notebooks/Chap03/3_3_Shallow_Network_Regions.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"The purpose of this notebook is to compute the maximum possible number of linear regions as seen in figure 3.9 of the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

2
Notebooks/Chap03/3_4_Activation_Functions.ipynb
View File

@@ -22,7 +22,7 @@
"\n",
"The purpose of this practical is to experiment with different activation functions. <br>\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and write code to complete the functions. There are also questions interspersed in the text.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
]

4
Notebooks/Chap04/4_1_Composing_Networks.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"The purpose of this notebook is to understand what happens when we feed one neural network into another. It works through an example similar to 4.1 and varies both networks\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions"
],
@@ -343,7 +343,7 @@
{
"cell_type": "code",
"source": [
"# TO DO\n",
"# TODO\n",
"# How many linear regions would there be if we ran N copies of the first network, feeding the result of the first\n",
"# into the second, the second into the third and so on, and then passed the result into the original second\n",
"# network (blue curve above)\n",

4
Notebooks/Chap04/4_2_Clipping_functions.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"The purpose of this notebook is to understand how a neural network with two hidden layers build more complicated functions by clipping and recombining the representations at the intermediate hidden variables.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions"
],
@@ -169,7 +169,7 @@
{
"cell_type": "code",
"source": [
"# Define parameters (note first dimension of theta and phi is padded to make indices match\n",
"# Define parameters (note first dimension of theta and psi is padded to make indices match\n",
"# notation in book)\n",
"theta = np.zeros([4,2])\n",
"psi = np.zeros([4,4])\n",

2
Notebooks/Chap04/4_3_Deep_Networks.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"This notebook investigates converting neural networks to matrix form.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

2
Notebooks/Chap05/5_1_Least_Squares_Loss.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"This notebook investigates the least squares loss and the equivalence of maximum likelihood and minimum negative log likelihood.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

2
Notebooks/Chap05/5_2_Binary_Cross_Entropy_Loss.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"This notebook investigates the binary cross-entropy loss. It follows from applying the formula in section 5.2 to a loss function based on the Bernoulli distribution.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

13
Notebooks/Chap05/5_3_Multiclass_Cross_entropy_Loss.ipynb
View File

@@ -20,7 +20,7 @@
"\n",
"This notebook investigates the multi-class cross-entropy loss. It follows from applying the formula in section 5.2 to a loss function based on the Categorical distribution.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
]
@@ -236,11 +236,10 @@
},
"outputs": [],
"source": [
"# Let's double check we get the right answer before proceeding\n",
"print(\"Correct answer = %3.3f, Your answer = %3.3f\"%(0.2,categorical_distribution(np.array([[0]]),np.array([[0.2],[0.5],[0.3]]))))\n",
"print(\"Correct answer = %3.3f, Your answer = %3.3f\"%(0.5,categorical_distribution(np.array([[1]]),np.array([[0.2],[0.5],[0.3]]))))\n",
"print(\"Correct answer = %3.3f, Your answer = %3.3f\"%(0.3,categorical_distribution(np.array([[2]]),np.array([[0.2],[0.5],[0.3]]))))\n",
"\n"
"# Here are three examples\n",
"print(categorical_distribution(np.array([[0]]),np.array([[0.2],[0.5],[0.3]])))\n",
"print(categorical_distribution(np.array([[1]]),np.array([[0.2],[0.5],[0.3]])))\n",
"print(categorical_distribution(np.array([[2]]),np.array([[0.2],[0.5],[0.3]])))"
]
},
{
@@ -460,4 +459,4 @@
},
"nbformat": 4,
"nbformat_minor": 0
}
}

2
Notebooks/Chap06/6_1_Line_Search.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"This notebook investigates how to find the minimum of a 1D function using line search as described in Figure 6.10.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
],

2
Notebooks/Chap06/6_2_Gradient_Descent.ipynb
View File

@@ -20,7 +20,7 @@
"\n",
"This notebook recreates the gradient descent algorithm as shown in figure 6.1.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
"\n"

2
Notebooks/Chap06/6_3_Stochastic_Gradient_Descent.ipynb
View File

@@ -20,7 +20,7 @@
"\n",
"This notebook investigates gradient descent and stochastic gradient descent and recreates figure 6.5 from the book\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
"\n",

2
Notebooks/Chap06/6_4_Momentum.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"This notebook investigates the use of momentum as illustrated in figure 6.7 from the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
"\n",

2
Notebooks/Chap06/6_5_Adam.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"This notebook investigates the Adam algorithm as illustrated in figure 6.9 from the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

2
Notebooks/Chap07/7_1_Backpropagation_in_Toy_Model.ipynb
View File

@@ -22,7 +22,7 @@
"\n",
"This notebook computes the derivatives of the toy function discussed in section 7.3 of the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
]

12
Notebooks/Chap07/7_2_Backpropagation.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"This notebook runs the backpropagation algorithm on a deep neural network as described in section 7.4 of the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],
@@ -248,23 +248,23 @@
"\n",
" # Now work backwards through the network\n",
" for layer in range(K,-1,-1):\n",
" # TODO Calculate the derivatives of the loss with respect to the biases at layer from all_dl_df[layer]. (eq 7.21)\n",
" # TODO Calculate the derivatives of the loss with respect to the biases at layer from all_dl_df[layer]. (eq 7.22)\n",
" # NOTE! To take a copy of matrix X, use Z=np.array(X)\n",
" # REPLACE THIS LINE\n",
" all_dl_dbiases[layer] = np.zeros_like(all_biases[layer])\n",
"\n",
" # TODO Calculate the derivatives of the loss with respect to the weights at layer from all_dl_df[layer] and all_h[layer] (eq 7.22)\n",
" # TODO Calculate the derivatives of the loss with respect to the weights at layer from all_dl_df[layer] and all_h[layer] (eq 7.23)\n",
" # Don't forget to use np.matmul\n",
" # REPLACE THIS LINE\n",
" all_dl_dweights[layer] = np.zeros_like(all_weights[layer])\n",
"\n",
" # TODO: calculate the derivatives of the loss with respect to the activations from weight and derivatives of next preactivations (second part of last line of eq 7.24)\n",
" # TODO: calculate the derivatives of the loss with respect to the activations from weight and derivatives of next preactivations (second part of last line of eq 7.25)\n",
" # REPLACE THIS LINE\n",
" all_dl_dh[layer] = np.zeros_like(all_h[layer])\n",
"\n",
"\n",
" if layer > 0:\n",
" # TODO Calculate the derivatives of the loss with respect to the pre-activation f (use derivative of ReLu function, first part of last line of eq. 7.24)\n",
" # TODO Calculate the derivatives of the loss with respect to the pre-activation f (use derivative of ReLu function, first part of last line of eq. 7.25)\n",
" # REPLACE THIS LINE\n",
" all_dl_df[layer-1] = np.zeros_like(all_f[layer-1])\n",
"\n",
@@ -352,4 +352,4 @@
"outputs": []
}
]
}
}

10
Notebooks/Chap07/7_3_Initialization.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"This notebook explores weight initialization in deep neural networks as described in section 7.5 of the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],
@@ -191,10 +191,10 @@
"# You can see that the values of the hidden units are increasing on average (the variance is across all hidden units at the layer\n",
"# and the 1000 training examples\n",
"\n",
"# TO DO\n",
"# TODO\n",
"# Change this to 50 layers with 80 hidden units per layer\n",
"\n",
"# TO DO\n",
"# TODO\n",
"# Now experiment with sigma_sq_omega to try to stop the variance of the forward computation exploding"
],
"metadata": {
@@ -340,10 +340,10 @@
"# You can see that the gradients of the hidden units are increasing on average (the standard deviation is across all hidden units at the layer\n",
"# and the 100 training examples\n",
"\n",
"# TO DO\n",
"# TODO\n",
"# Change this to 50 layers with 80 hidden units per layer\n",
"\n",
"# TO DO\n",
"# TODO\n",
"# Now experiment with sigma_sq_omega to try to stop the variance of the gradients exploding\n"
],
"metadata": {

8
Notebooks/Chap08/8_1_MNIST_1D_Performance.ipynb
View File

@@ -20,7 +20,7 @@
"\n",
"This notebook runs a simple neural network on the MNIST1D dataset as in figure 8.2a. It uses code from https://github.com/greydanus/mnist1d to generate the data.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
]
@@ -91,7 +91,7 @@
"D_i = 40 # Input dimensions\n",
"D_k = 100 # Hidden dimensions\n",
"D_o = 10 # Output dimensions\n",
"# TO DO:\n",
"# TODO:\n",
"# Define a model with two hidden layers of size 100\n",
"# And ReLU activations between them\n",
"# Replace this line (see Figure 7.8 of book for help):\n",
@@ -99,7 +99,7 @@
"\n",
"\n",
"def weights_init(layer_in):\n",
" # TO DO:\n",
" # TODO:\n",
" # Initialize the parameters with He initialization\n",
" # Replace this line (see figure 7.8 of book for help)\n",
" print(\"Initializing layer\")\n",
@@ -208,7 +208,7 @@
"id": "q-yT6re6GZS4"
},
"source": [
"**TO DO**\n",
"**TODO**\n",
"\n",
"Play with the model -- try changing the number of layers, hidden units, learning rate, batch size, momentum or anything else you like. See if you can improve the test results.\n",
"\n",

2
Notebooks/Chap08/8_2_Bias_Variance_Trade_Off.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"This notebook investigates the bias-variance trade-off for the toy model used throughout chapter 8 and reproduces the bias/variance trade off curves seen in figure 8.9.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

4
Notebooks/Chap08/8_3_Double_Descent.ipynb
View File

@@ -36,7 +36,7 @@
"\n",
"It uses the MNIST-1D database which can be found at https://github.com/greydanus/mnist1d\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],
@@ -217,7 +217,7 @@
"source": [
"The following code produces the double descent curve by training the model with different numbers of hidden units and plotting the test error.\n",
"\n",
"TO DO:\n",
"TODO:\n",
"\n",
"*Before* you run the code, and considering that there are 4000 training examples predict:<br>\n",
"\n",

2
Notebooks/Chap08/8_4_High_Dimensional_Spaces.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook investigates the strange properties of high-dimensional spaces as discussed in the notes at the end of chapter 8.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

2
Notebooks/Chap09/9_1_L2_Regularization.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook investigates adding L2 regularization to the loss function for the Gabor model as in figure 9.1.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
],

2
Notebooks/Chap09/9_2_Implicit_Regularization.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"This notebook investigates how the finite step sizes in gradient descent cause the trajectory to deviate and how this can be explained by adding an implicit regularization term. It recreates figure 9.3 from the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
],

2
Notebooks/Chap09/9_3_Ensembling.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook investigates how ensembling can improve the performance of models. We'll work with the simplified neural network model (figure 8.4 of book) which we can fit in closed form, and so we can eliminate any errors due to not finding the global maximum.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
],

2
Notebooks/Chap09/9_4_Bayesian_Approach.ipynb
View File

@@ -20,7 +20,7 @@
"\n",
"This notebook investigates the Bayesian approach to model fitting and reproduces figure 9.11 from the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
]

2
Notebooks/Chap09/9_5_Augmentation.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"This notebook investigates data augmentation for the MNIST-1D model.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
],

2
Notebooks/Chap10/10_1_1D_Convolution.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook investigates 1D convolutional layers.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
],

2
Notebooks/Chap10/10_2_Convolution_for_MNIST_1D.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook investigates a 1D convolutional network for MNIST-1D as in figure 10.7 and 10.8a.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
"\n"

2
Notebooks/Chap10/10_3_2D_Convolution.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook investigates the 2D convolution operation. It asks you to hand code the convolution so we can be sure that we are computing the same thing as in PyTorch. The next notebook uses the convolutional layers in PyTorch directly.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

2
Notebooks/Chap10/10_4_Downsampling_and_Upsampling.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook investigates the upsampling and downsampling methods discussed in section 10.4 of the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
],

2
Notebooks/Chap10/10_5_Convolution_For_MNIST.ipynb
View File

@@ -35,7 +35,7 @@
"\n",
"The code is adapted from https://nextjournal.com/gkoehler/pytorch-mnist\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
],

2
Notebooks/Chap11/11_1_Shattered_Gradients.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook investigates the phenomenon of shattered gradients as discussed in section 11.1.1. It replicates some of the experiments in [Balduzzi et al. (2017)](https://arxiv.org/abs/1702.08591).\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

2
Notebooks/Chap11/11_2_Residual_Networks.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook adapts the networks for MNIST1D to use residual connections.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
"\n"

2
Notebooks/Chap11/11_3_Batch_Normalization.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook investigates the use of batch normalization in residual networks.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
"\n"

2
Notebooks/Chap12/12_1_Self_Attention.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook builds a self-attention mechanism from scratch, as discussed in section 12.2 of the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
"\n"

2
Notebooks/Chap12/12_2_Multihead_Self_Attention.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"This notebook builds a multihead self-attention mechanism as in figure 12.6\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
"\n"

2
Notebooks/Chap12/12_3_Tokenization.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook builds set of tokens from a text string as in figure 12.8 of the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"I adapted this code from *SOMEWHERE*. If anyone recognizes it, can you let me know and I will give the proper attribution or rewrite if the license is not permissive.\n",
"\n",

2
Notebooks/Chap12/12_4_Decoding_Strategies.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This practical investigates neural decoding from transformer models. \n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

2
Notebooks/Chap13/13_1_Graph_Representation.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook investigates representing graphs with matrices as illustrated in figure 13.4 from the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
"\n"

2
Notebooks/Chap13/13_2_Graph_Classification.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"This notebook investigates representing graphs with matrices as illustrated in figure 13.4 from the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

2
Notebooks/Chap13/13_3_Neighborhood_Sampling.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook investigates neighborhood sampling of graphs as in figure 13.10 from the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

2
Notebooks/Chap13/13_4_Graph_Attention_Networks.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook builds a graph attention mechanism from scratch, as discussed in section 13.8.6 of the book and illustrated in figure 13.12c\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
"\n"

2
Notebooks/Chap15/15_1_GAN_Toy_Example.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"This notebook investigates the GAN toy example as illustrated in figure 15.1 in the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

2
Notebooks/Chap15/15_2_Wasserstein_Distance.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"This notebook investigates the GAN toy example as illustrated in figure 15.1 in the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

2
Notebooks/Chap16/16_1_1D_Normalizing_Flows.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook investigates a 1D normalizing flows example similar to that illustrated in figures 16.1 to 16.3 in the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

2
Notebooks/Chap16/16_2_Autoregressive_Flows.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook investigates a 1D normalizing flows example similar to that illustrated in figure 16.7 in the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

2
Notebooks/Chap16/16_3_Contraction_Mappings.ipynb
View File

@@ -22,7 +22,7 @@
"\n",
"This notebook investigates a 1D normalizing flows example similar to that illustrated in figure 16.9 in the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
]

2
Notebooks/Chap17/17_1_Latent_Variable_Models.ipynb
View File

@@ -22,7 +22,7 @@
"\n",
"This notebook investigates a non-linear latent variable model similar to that in figures 17.2 and 17.3 of the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
]

2
Notebooks/Chap17/17_2_Reparameterization_Trick.ipynb
View File

@@ -20,7 +20,7 @@
"\n",
"This notebook investigates the reparameterization trick as described in section 17.7 of the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
]

36
Notebooks/Chap17/17_3_Importance_Sampling.ipynb
View File

@@ -1,18 +1,16 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "view-in-github"
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/Notebooks/Chap17/17_3_Importance_Sampling.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "t9vk9Elugvmi"
@@ -22,7 +20,7 @@
"\n",
"This notebook investigates importance sampling as described in section 17.8.1 of the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
]
@@ -40,7 +38,6 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "f7a6xqKjkmvT"
@@ -126,7 +123,6 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "Jr4UPcqmnXCS"
@@ -166,8 +162,8 @@
"mean_all = np.zeros_like(n_sample_all)\n",
"variance_all = np.zeros_like(n_sample_all)\n",
"for i in range(len(n_sample_all)):\n",
" print(\"Computing mean and variance for expectation with %d samples\"%(n_sample_all[i]))\n",
" mean_all[i],variance_all[i] = compute_mean_variance(n_sample_all[i])"
" mean_all[i],variance_all[i] = compute_mean_variance(n_sample_all[i])\n",
" print(\"No samples: \", n_sample_all[i], \", Mean: \", mean_all[i], \", Variance: \", variance_all[i])"
]
},
{
@@ -189,7 +185,6 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "XTUpxFlSuOl7"
@@ -199,7 +194,6 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "6hxsl3Pxo1TT"
@@ -234,7 +228,6 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "G9Xxo0OJsIqD"
@@ -283,7 +276,6 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "2sVDqP0BvxqM"
@@ -313,8 +305,8 @@
"mean_all2 = np.zeros_like(n_sample_all)\n",
"variance_all2 = np.zeros_like(n_sample_all)\n",
"for i in range(len(n_sample_all)):\n",
" print(\"Computing variance for expectation with %d samples\"%(n_sample_all[i]))\n",
" mean_all2[i], variance_all2[i] = compute_mean_variance2(n_sample_all[i])"
" mean_all2[i], variance_all2[i] = compute_mean_variance2(n_sample_all[i])\n",
" print(\"No samples: \", n_sample_all[i], \", Mean: \", mean_all2[i], \", Variance: \", variance_all2[i])"
]
},
{
@@ -348,7 +340,6 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "EtBP6NeLwZqz"
@@ -360,7 +351,6 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "_wuF-NoQu1--"
@@ -432,8 +422,8 @@
"mean_all2b = np.zeros_like(n_sample_all)\n",
"variance_all2b = np.zeros_like(n_sample_all)\n",
"for i in range(len(n_sample_all)):\n",
" print(\"Computing variance for expectation with %d samples\"%(n_sample_all[i]))\n",
" mean_all2b[i], variance_all2b[i] = compute_mean_variance2b(n_sample_all[i])"
" mean_all2b[i], variance_all2b[i] = compute_mean_variance2b(n_sample_all[i])\n",
" print(\"No samples: \", n_sample_all[i], \", Mean: \", mean_all2b[i], \", Variance: \", variance_all2b[i])"
]
},
{
@@ -478,7 +468,6 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "y8rgge9MNiOc"
@@ -490,9 +479,8 @@
],
"metadata": {
"colab": {
"authorship_tag": "ABX9TyNecz9/CDOggPSmy1LjT/Dv",
"include_colab_link": true,
"provenance": []
"provenance": [],
"include_colab_link": true
},
"kernelspec": {
"display_name": "Python 3",
@@ -504,4 +492,4 @@
},
"nbformat": 4,
"nbformat_minor": 0
}
}

2
Notebooks/Chap18/18_1_Diffusion_Encoder.ipynb
View File

@@ -20,7 +20,7 @@
"\n",
"This notebook investigates the diffusion encoder as described in section 18.2 of the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
]

2
Notebooks/Chap18/18_2_1D_Diffusion_Model.ipynb
View File

@@ -22,7 +22,7 @@
"\n",
"This notebook investigates the diffusion encoder as described in section 18.3 and 18.4 of the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
]

2
Notebooks/Chap18/18_3_Reparameterized_Model.ipynb
View File

@@ -22,7 +22,7 @@
"\n",
"This notebook investigates the reparameterized model as described in section 18.5 of the book and implements algorithms 18.1 and 18.2.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
]

2
Notebooks/Chap18/18_4_Families_of_Diffusion_Models.ipynb
View File

@@ -22,7 +22,7 @@
"\n",
"This notebook investigates the reparameterized model as described in section 18.5 of the book and computers the results shown in figure 18.10c-f. These models are based on the paper \"Denoising diffusion implicit models\" which can be found [here](https://arxiv.org/pdf/2010.02502.pdf).\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
]

2
Notebooks/Chap19/19_1_Markov_Decision_Processes.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook investigates Markov decision processes as described in section 19.1 of the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

9
Notebooks/Chap19/19_2_Dynamic_Programming.ipynb
View File

@@ -4,7 +4,6 @@
"metadata": {
"colab": {
"provenance": [],
"authorship_tag": "ABX9TyOlD6kmCxX3SKKuh3oJikKA",
"include_colab_link": true
},
"kernelspec": {
@@ -33,7 +32,7 @@
"\n",
"This notebook investigates the dynamic programming approach to tabular reinforcement learning as described in figure 19.10 of the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],
@@ -406,6 +405,10 @@
" state_values_new[state] = 3.0\n",
" break\n",
"\n",
" # TODO -- Write this function (from equation 19.11, but bear in mind policy is deterministic here)\n",
" # Replace this line\n",
" state_values_new[state] = 0\n",
"\n",
" return state_values_new\n",
"\n",
"# Greedily choose the action that maximizes the value for each state.\n",
@@ -527,4 +530,4 @@
}
}
]
}
}

2
Notebooks/Chap19/19_3_Monte_Carlo_Methods.ipynb
View File

@@ -22,7 +22,7 @@
"\n",
"NOTE! There is a mistake in Figure 19.11 in the first printing of the book, so check the errata to avoid becoming confused. Apologies!\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
"\n",

2
Notebooks/Chap19/19_4_Temporal_Difference_Methods.ipynb
View File

@@ -20,7 +20,7 @@
"\n",
"This notebook investigates temporal difference methods for tabular reinforcement learning as described in section 19.3.3 of the book\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
"\n",

2
Notebooks/Chap19/19_5_Control_Variates.ipynb
View File

@@ -34,7 +34,7 @@
"This notebook investigates the method of control variates as described in figure 19.16\n",
"\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

2
Notebooks/Chap20/20_1_Random_Data.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook investigates training the network with random data, as illustrated in figure 20.1.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
"\n"

2
Notebooks/Chap20/20_2_Full_Batch_Gradient_Descent.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook investigates training a network with full batch gradient descent as in figure 20.2. There is also a version (notebook takes a long time to run), but this didn't speed it up much for me. If you run out of CoLab time, you'll need to download the Python file and run locally.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
],

2
Notebooks/Chap20/20_2_Full_Batch_Gradient_Descent_GPU.ipynb
View File

@@ -35,7 +35,7 @@
"\n",
"This notebook investigates training a network with full batch gradient descent as in figure 20.2. This is the GPU version (notebook takes a long time to run). If you are using Colab then you need to go change the runtime type to GPU on the Runtime menu. Even then, you may run out of time. If that's the case, you'll need to download the Python file and run locally.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n",
"\n"

2
Notebooks/Chap20/20_3_Lottery_Tickets.ipynb
View File

@@ -32,7 +32,7 @@
"\n",
"This notebook investigates the phenomenon of lottery tickets as discussed in section 20.2.7. This notebook is highly derivative of the MNIST-1D code hosted by Sam Greydanus at https://github.com/greydanus/mnist1d. \n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions."
]

2
Notebooks/Chap20/20_4_Adversarial_Attacks.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook builds uses the network for classification of MNIST from Notebook 10.5. The code is adapted from https://nextjournal.com/gkoehler/pytorch-mnist, and uses the fast gradient sign attack of [Goodfellow et al. (2015)](https://arxiv.org/abs/1412.6572). Having trained, the network, we search for adversarial examples -- inputs which look very similar to class A, but are mistakenly classified as class B. We do this by starting with a correctly classified example and perturbing it according to the gradients of the network so that the output changes.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
],

2
Notebooks/Chap21/21_1_Bias_Mitigation.ipynb
View File

@@ -22,7 +22,7 @@
"\n",
"This notebook investigates a post-processing method for bias mitigation (see figure 21.2 in the book). It based on this [blog](https://www.borealisai.com/research-blogs/tutorial1-bias-and-fairness-ai/) that I wrote for Borealis AI in 2019, which itself was derived from [this blog](https://research.google.com/bigpicture/attacking-discrimination-in-ml/) by Wattenberg, Viégas, and Hardt.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
]

2
Notebooks/Chap21/21_2_Explainability.ipynb
View File

@@ -33,7 +33,7 @@
"\n",
"This notebook investigates the LIME explainability method as depicted in figure 21.3 of the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TODO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
"Contact me at udlbookmail@gmail.com if you find any mistakes or have any suggestions.\n"
],

326
Trees/LinearRegression_FitModel.ipynb Normal file
View File

File diff suppressed because one or more lines are too long

357
Trees/LinearRegression_FitModel_Answers.ipynb Normal file
View File

File diff suppressed because one or more lines are too long

343
Trees/LinearRegression_FitModel_Quadratic.ipynb Normal file
View File

File diff suppressed because one or more lines are too long

51
Trees/LinearRegression_LeastSquares.ipynb
View File

@@ -1,51 +0,0 @@
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": [],
"authorship_tag": "ABX9TyM1pe3HkxLrjbeKezq1MlM5",
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/Trees/LinearRegression_LeastSquares.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"source": [
"# Least Squares Loss"
],
"metadata": {
"id": "uORlKyPv02ge"
}
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "bbF6SE_F0tU8"
},
"outputs": [],
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt"
]
}
]
}

232
Trees/LinearRegression_LossFunction.ipynb
View File

File diff suppressed because one or more lines are too long

325
Trees/LinearRegression_LossFunction_Answers.ipynb Normal file
View File

File diff suppressed because one or more lines are too long

248
Trees/SAT_Exhaustive.ipynb Normal file
View File

File diff suppressed because one or more lines are too long

250
Trees/SAT_Exhaustive_Answers.ipynb Normal file
View File

File diff suppressed because one or more lines are too long

251
Trees/SAT_Tseitin.ipynb Normal file
View File

File diff suppressed because one or more lines are too long

310
Trees/SAT_Tseitin_Answers.ipynb Normal file
View File

File diff suppressed because one or more lines are too long

BIN
UDL_Answer_Booklet_Students.pdf
View File

Binary file not shown.

BIN
UDL_Errata.pdf
View File

Binary file not shown.