dr_domi/udlbook
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Created using Colaboratory

Browse Source
This commit is contained in:
udlbook
2023-11-14 10:34:50 +00:00
parent 00d180bf88
commit f63ddc7313
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
Notebooks/Chap07/7_1_Backpropagation_in_Toy_Model.ipynb
View File

@@ -4,7 +4,7 @@
"metadata": { "metadata": {
"colab": { "colab": {
"provenance": [], "provenance": [],
"authorship_tag": "ABX9TyP5wHK5E7/el+vxU947K3q8", "authorship_tag": "ABX9TyPmaYT+KLPAmZxse087Eisa",
"include_colab_link": true "include_colab_link": true
}, },
"kernelspec": { "kernelspec": {
@@ -281,7 +281,7 @@
"\\frac{\\partial \\ell_i}{\\partial h_{3}} =\\frac{\\partial f_{3}}{\\partial h_{3}} \\frac{\\partial \\ell_i}{\\partial f_{3}} .\n", "\\frac{\\partial \\ell_i}{\\partial h_{3}} =\\frac{\\partial f_{3}}{\\partial h_{3}} \\frac{\\partial \\ell_i}{\\partial f_{3}} .\n",
"\\end{equation}\n", "\\end{equation}\n",
"\n", "\n",
"The left-hand side asks how $\\ell_i$ changes when $h_{3}$ changes. The right-hand side says we can decompose this into (i) how $ell_i$ changes when $f_{3}$ changes and how $f_{3}$ changes when $h_{3}$ changes. So you get a chain of events happening: $h_{3}$ changes $f_{3}$, which changes $\\ell_i$, and the derivatives represent the effects of this chain. Notice that we computed the first of these derivatives already and is $2 (f_3-y)$. We calculated $f_{3}$ in step 1. The second term is the derivative of $\\beta_{3} + \\omega_{3}h_{3}$ with respect to $h_3$ which is simply $\\omega_3$. \n", "The left-hand side asks how $\\ell_i$ changes when $h_{3}$ changes. The right-hand side says we can decompose this into (i) how $\\ell_i$ changes when $f_{3}$ changes and how $f_{3}$ changes when $h_{3}$ changes. So you get a chain of events happening: $h_{3}$ changes $f_{3}$, which changes $\\ell_i$, and the derivatives represent the effects of this chain. Notice that we computed the first of these derivatives already and is $2 (f_3-y)$. We calculated $f_{3}$ in step 1. The second term is the derivative of $\\beta_{3} + \\omega_{3}h_{3}$ with respect to $h_3$ which is simply $\\omega_3$. \n",
"\n", "\n",
"We can continue in this way, computing the derivatives of the output with respect to these intermediate quantities:\n", "We can continue in this way, computing the derivatives of the output with respect to these intermediate quantities:\n",
"\n", "\n",