dr_domi/udlbook
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #164 from yrahal/main

Browse Source 
Fix minor typos in Chap07 notebooks
This commit is contained in:
udlbook
2024-03-25 16:43:55 -04:00
committed by GitHub
parent c0cd9c2aea d0d413b9f6
commit 4a5bd9c4d5
3 changed files with 15 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
Notebooks/Chap07/7_1_Backpropagation_in_Toy_Model.ipynb
View File

@@ -279,7 +279,7 @@
"f2: true value = 7.137, your value = 0.000\n",
"h3: true value = 0.657, your value = 0.000\n",
"f3: true value = 2.372, your value = 0.000\n",
"like original = 0.139, like from forward pass = 0.000\n"
"l_i original = 0.139, l_i from forward pass = 0.000\n"
]
}
],
@@ -292,7 +292,7 @@
"print(\"f2: true value = %3.3f, your value = %3.3f\"%(7.137, f2))\n",
"print(\"h3: true value = %3.3f, your value = %3.3f\"%(0.657, h3))\n",
"print(\"f3: true value = %3.3f, your value = %3.3f\"%(2.372, f3))\n",
"print(\"like original = %3.3f, like from forward pass = %3.3f\"%(l_i_func, l_i))\n"
"print(\"l_i original = %3.3f, l_i from forward pass = %3.3f\"%(l_i_func, l_i))\n"
]
},
{

14
Notebooks/Chap07/7_2_Backpropagation.ipynb
View File

@@ -115,9 +115,9 @@
{
"cell_type": "markdown",
"source": [
"Now let's run our random network. The weight matrices $\\boldsymbol\\Omega_{1\\ldots K}$ are the entries of the list \"all_weights\" and the biases $\\boldsymbol\\beta_{1\\ldots k}$ are the entries of the list \"all_biases\"\n",
"Now let's run our random network. The weight matrices $\\boldsymbol\\Omega_{1\\ldots K}$ are the entries of the list \"all_weights\" and the biases $\\boldsymbol\\beta_{1\\ldots K}$ are the entries of the list \"all_biases\"\n",
"\n",
"We know that we will need the activations $\\mathbf{f}_{0\\ldots K}$ and the activations $\\mathbf{h}_{1\\ldots K}$ for the forward pass of backpropagation, so we'll store and return these as well.\n"
"We know that we will need the preactivations $\\mathbf{f}_{0\\ldots K}$ and the activations $\\mathbf{h}_{1\\ldots K}$ for the forward pass of backpropagation, so we'll store and return these as well.\n"
],
"metadata": {
"id": "5irtyxnLJSGX"
@@ -132,7 +132,7 @@
" K = len(all_weights) -1\n",
"\n",
" # We'll store the pre-activations at each layer in a list \"all_f\"\n",
" # and the activations in a second list[all_h].\n",
" # and the activations in a second list \"all_h\".\n",
" all_f = [None] * (K+1)\n",
" all_h = [None] * (K+1)\n",
"\n",
@@ -143,7 +143,7 @@
" # Run through the layers, calculating all_f[0...K-1] and all_h[1...K]\n",
" for layer in range(K):\n",
" # Update preactivations and activations at this layer according to eqn 7.16\n",
" # Remmember to use np.matmul for matrrix multiplications\n",
" # Remmember to use np.matmul for matrix multiplications\n",
" # TODO -- Replace the lines below\n",
" all_f[layer] = all_h[layer]\n",
" all_h[layer+1] = all_f[layer]\n",
@@ -166,7 +166,7 @@
{
"cell_type": "code",
"source": [
"# Define in input\n",
"# Define input\n",
"net_input = np.ones((D_i,1)) * 1.2\n",
"# Compute network output\n",
"net_output, all_f, all_h = compute_network_output(net_input,all_weights, all_biases)\n",
@@ -249,7 +249,7 @@
"\n",
" # Now work backwards through the network\n",
" for layer in range(K,-1,-1):\n",
" # TODO Calculate the derivatives of the loss with respect to the biases at layer this from all_dl_df[layer]. (eq 7.21)\n",
" # TODO Calculate the derivatives of the loss with respect to the biases at layer from all_dl_df[layer]. (eq 7.21)\n",
" # NOTE! To take a copy of matrix X, use Z=np.array(X)\n",
" # REPLACE THIS LINE\n",
" all_dl_dbiases[layer] = np.zeros_like(all_biases[layer])\n",
@@ -265,7 +265,7 @@
"\n",
"\n",
" if layer > 0:\n",
" # TODO Calculate the derivatives of the loss with respect to the pre-activation f (use deriv of ReLu function, first part of last line of eq. 7.24)\n",
" # TODO Calculate the derivatives of the loss with respect to the pre-activation f (use derivative of ReLu function, first part of last line of eq. 7.24)\n",
" # REPLACE THIS LINE\n",
" all_dl_df[layer-1] = np.zeros_like(all_f[layer-1])\n",
"\n",

6
Notebooks/Chap07/7_3_Initialization.ipynb
View File

@@ -120,7 +120,7 @@
" K = len(all_weights)-1\n",
"\n",
" # We'll store the pre-activations at each layer in a list \"all_f\"\n",
" # and the activations in a second list[all_h].\n",
" # and the activations in a second list \"all_h\".\n",
" all_f = [None] * (K+1)\n",
" all_h = [None] * (K+1)\n",
"\n",
@@ -151,7 +151,7 @@
{
"cell_type": "markdown",
"source": [
"Now let's investigate how this the size of the outputs vary as we change the initialization variance:\n"
"Now let's investigate how the size of the outputs vary as we change the initialization variance:\n"
],
"metadata": {
"id": "bIUrcXnOqChl"
@@ -196,7 +196,7 @@
"# Change this to 50 layers with 80 hidden units per layer\n",
"\n",
"# TO DO\n",
"# Now experiment with sigma_sq_omega to try to stop the variance of the forward computation explode"
"# Now experiment with sigma_sq_omega to try to stop the variance of the forward computation exploding"
],
"metadata": {
"id": "VL_SO4tar3DC"