dr_domi/udlbook
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Created using Colaboratory

Browse Source
This commit is contained in:
udlbook
2023-10-24 13:10:47 +01:00
parent 0af1ea979c
commit 33a94fb59f
1 changed files with 8 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
Notebooks/Chap05/5_1_Least_Squares_Loss.ipynb
View File

@@ -4,7 +4,7 @@
"metadata": {
"colab": {
"provenance": [],
"authorship_tag": "ABX9TyPX88BLalmJTle9GSAZMJcz",
"authorship_tag": "ABX9TyPDNteVt0SjrR97OiS386NZ",
"include_colab_link": true
},
"kernelspec": {
@@ -306,7 +306,8 @@
"source": [
"# Return the negative log likelihood of the data under the model\n",
"def compute_negative_log_likelihood(y_train, mu, sigma):\n",
" # TODO -- compute the likelihood of the data -- don't use the likelihood function above -- compute the negative sum of the log probabilities\n",
" # TODO -- compute the negative log likelihood of the data without using aproduct\n",
" # In other words, compute minus one times the sum of the log probabilities\n",
" # Equation 5.4 in the notes\n",
" # You will need np.sum(), np.log()\n",
" # Replace the line below\n",
@@ -352,7 +353,7 @@
{
"cell_type": "code",
"source": [
"# Return the squared distance between the predicted\n",
"# Return the squared distance between the observed data (y_train) and the prediction of the model (y_pred)\n",
"def compute_sum_of_squares(y_train, y_pred):\n",
" # TODO -- compute the sum of squared distances between the training data and the model prediction\n",
" # Eqn 5.10 in the notes. Make sure that you understand this, and ask questions if you don't\n",
@@ -372,9 +373,9 @@
"source": [
"# Let's test this again\n",
"beta_0, omega_0, beta_1, omega_1 = get_parameters()\n",
"# Use our neural network to predict the mean of the Gaussian\n",
"y_pred = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
"# Compute the log likelihood\n",
"# Use our neural network to predict the mean of the Gaussian, which is out best prediction of y\n",
"y_pred = mu_pred = shallow_nn(x_train, beta_0, omega_0, beta_1, omega_1)\n",
"# Compute the sum of squares\n",
"sum_of_squares = compute_sum_of_squares(y_train, y_pred)\n",
"# Let's double check we get the right answer before proceeding\n",
"print(\"Correct answer = %9.9f, Your answer = %9.9f\"%(2.020992572,sum_of_squares))"
@@ -554,7 +555,7 @@
{
"cell_type": "markdown",
"source": [
"Obviously, to fit the full neural model we would vary all of the 10 parameters of the network in the $\\boldsymbol\\beta_{0},\\boldsymbol\\omega_{0},\\boldsymbol\\beta_{1},\\boldsymbol\\omega_{1}$ (and maybe $\\sigma$) until we find the combination that have the maximum likelihood / minimum negative log likelihood / least squares.<br><br>\n",
"Obviously, to fit the full neural model we would vary all of the 10 parameters of the network in $\\boldsymbol\\beta_{0},\\boldsymbol\\omega_{0},\\boldsymbol\\beta_{1},\\boldsymbol\\omega_{1}$ (and maybe $\\sigma$) until we find the combination that have the maximum likelihood / minimum negative log likelihood / least squares.<br><br>\n",
"\n",
"Here we just varied one at a time as it is easier to see what is going on. This is known as **coordinate descent**.\n"
],