dr_domi/udlbook
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'main' into main

Browse Source
This commit is contained in:
udlbook
2023-12-10 17:11:38 +00:00
committed by GitHub
parent aea371dc7d aacf54fb8b
commit 16ef8a7333
26 changed files with 87 additions and 65 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
Notebooks/Chap01/1_1_BackgroundMathematics.ipynb
View File

@@ -83,7 +83,7 @@
"source": [
"# Plot the 1D linear function\n",
"\n",
"# Define an array of x values from 0 to 10 with increments of 0.1\n",
"# Define an array of x values from 0 to 10 with increments of 0.01\n",
"# https://numpy.org/doc/stable/reference/generated/numpy.arange.html\n",
"x = np.arange(0.0,10.0, 0.01)\n",
"# Compute y using the function you filled in above\n",
@@ -171,7 +171,7 @@
"# Color represents y value (brighter = higher value)\n",
"# Black = -10 or less, White = +10 or more\n",
"# 0 = mid orange\n",
"# Lines are conoturs where value is equal\n",
"# Lines are contours where value is equal\n",
"draw_2D_function(x1,x2,y)\n",
"\n",
"# TODO\n",
@@ -308,7 +308,7 @@
"source": [
"# Draw the exponential function\n",
"\n",
"# Define an array of x values from -5 to 5 with increments of 0.1\n",
"# Define an array of x values from -5 to 5 with increments of 0.01\n",
"x = np.arange(-5.0,5.0, 0.01)\n",
"y = np.exp(x) ;\n",
"\n",
@@ -354,7 +354,7 @@
"source": [
"# Draw the logarithm function\n",
"\n",
"# Define an array of x values from -5 to 5 with increments of 0.1\n",
"# Define an array of x values from -5 to 5 with increments of 0.01\n",
"x = np.arange(0.01,5.0, 0.01)\n",
"y = np.log(x) ;\n",
"\n",

2
Notebooks/Chap03/3_2_Shallow_Networks_II.ipynb
View File

@@ -182,7 +182,7 @@
{
"cell_type": "markdown",
"source": [
"Now we'll extend this model to have two outputs $y_1$ and $y_2$, each of which can be visualized with a separate heatmap. You will now have sets of parameters $\\phi_{10}, \\phi_{11},\\phi_{12}$ and $\\phi_{20}, \\phi_{21},\\phi_{22}$ that correspond to each of these outputs."
"Now we'll extend this model to have two outputs $y_1$ and $y_2$, each of which can be visualized with a separate heatmap. You will now have sets of parameters $\\phi_{10}, \\phi_{11}, \\phi_{12}$, $\\phi_{13} and $\\phi_{20}, \\phi_{21}, \\phi_{22}$, \\phi_{23}$ that correspond to each of these outputs."
],
"metadata": {
"id": "Xl6LcrUyM7Lh"

4
Notebooks/Chap03/3_3_Shallow_Network_Regions.ipynb
View File

@@ -79,7 +79,7 @@
"source": [
"def number_regions(Di, D):\n",
" # TODO -- implement Zaslavsky's formula\n",
" # You can use math.com() https://www.w3schools.com/python/ref_math_comb.asp\n",
" # You can use math.comb() https://www.w3schools.com/python/ref_math_comb.asp\n",
" # Replace this code\n",
" N = 1;\n",
"\n",
@@ -256,4 +256,4 @@
"outputs": []
}
]
}
}

15
Notebooks/Chap10/10_1_1D_Convolution.ipynb
View File

@@ -4,7 +4,7 @@
"metadata": {
"colab": {
"provenance": [],
"authorship_tag": "ABX9TyPTidpnPhn4O5QF011gt0cz",
"authorship_tag": "ABX9TyML7rfAGE4gvmNUEiK5x3PS",
"include_colab_link": true
},
"kernelspec": {
@@ -41,6 +41,17 @@
"id": "el8l05WQEO46"
}
},
{
"cell_type": "markdown",
"source": [
"NOTE!!\n",
"\n",
"If you have the first edition of the printed book, it mistakenly refers to a convolutional filter with no spaces between the elements (i.e. a normal filter without dilation) as having dilation zero. Actually, the convention is (weirdly) that this has dilation one. And when there is one space between the elements, this is dilation two. This notebook reflects the correct convention and so will be out of sync with the printed book. If this is confusing, check the [errata](https://github.com/udlbook/udlbook/blob/main/UDL_Errata.pdf) document."
],
"metadata": {
"id": "ggQrHkFZcUiV"
}
},
{
"cell_type": "code",
"source": [
@@ -50,7 +61,7 @@
"metadata": {
"id": "nw7k5yCtOzoK"
},
"execution_count": null,
"execution_count": 1,
"outputs": []
},
{

5
Notebooks/Chap10/10_2_Convolution_for_MNIST_1D.ipynb
View File

@@ -4,7 +4,7 @@
"metadata": {
"colab": {
"provenance": [],
"authorship_tag": "ABX9TyN1v/yg9PtdSVOWlYJ7bgkz",
"authorship_tag": "ABX9TyN1qtywBuyezaVMnc9MI7x2",
"include_colab_link": true
},
"kernelspec": {
@@ -141,6 +141,9 @@
"# https://pytorch.org/docs/stable/generated/torch.nn.Flatten.html\n",
"# https://pytorch.org/docs/1.13/generated/torch.nn.Linear.html?highlight=linear#torch.nn.Linear\n",
"\n",
"# NOTE THAT THE CONVOLUTIONAL LAYERS NEED TO TAKE THE NUMBER OF INPUT CHANNELS AS A PARAMETER\n",
"# AND NOT THE INPUT SIZE.\n",
"\n",
"# Replace the following function:\n",
"model = nn.Sequential(\n",
"nn.Flatten(),\n",

7
Notebooks/Chap10/10_3_2D_Convolution.ipynb
View File

@@ -4,7 +4,7 @@
"metadata": {
"colab": {
"provenance": [],
"authorship_tag": "ABX9TyMmbD0cKYvIHXbKX4AupA1x",
"authorship_tag": "ABX9TyNDaU2KKZDyY9Ea7vm/fNxo",
"include_colab_link": true
},
"kernelspec": {
@@ -114,6 +114,11 @@
" # Create output\n",
" out = np.zeros((batchSize, channelsOut, imageHeightOut, imageWidthOut), dtype=np.float32)\n",
"\n",
" # !!!!!! NOTE THERE IS A SUBTLETY HERE !!!!!!!!\n",
" # I have padded the image with zeros above, so it is surrouned by a \"ring\" of zeros\n",
" # That means that the image indexes are all off by one\n",
" # This actually makes your code simpler\n",
"\n",
" for c_y in range(imageHeightOut):\n",
" for c_x in range(imageWidthOut):\n",
" for c_kernel_y in range(kernelHeight):\n",

2
Notebooks/Chap12/12_1_Self_Attention.ipynb
View File

@@ -31,7 +31,7 @@
"source": [
"# **Notebook 12.1: Self Attention**\n",
"\n",
"This notebook builds a self-attnetion mechanism from scratch, as discussed in section 12.2 of the book.\n",
"This notebook builds a self-attention mechanism from scratch, as discussed in section 12.2 of the book.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",

2
Notebooks/Chap18/18_1_Diffusion_Encoder.ipynb
View File

@@ -407,7 +407,7 @@
" # 1. For each x (value in x_plot_vals):\n",
" # 2. Compute the mean and variance of the diffusion kernel at time t\n",
" # 3. Compute pdf of this Gaussian at every x_plot_val\n",
" # 4. Weight Gaussian by probability at position x and by 0.01 to compensate for bin size\n",
" # 4. Weight Gaussian by probability at position x and by 0.01 to componensate for bin size\n",
" # 5. Accumulate weighted Gaussian in marginal at time t.\n",
" # 6. Multiply result by 0.01 to compensate for bin size\n",
" # Replace this line:\n",

8
Notebooks/Chap21/21_1_Bias_Mitigation.ipynb
View File

@@ -31,7 +31,7 @@
"source": [
"# **Notebook 21.1: Bias mitigation**\n",
"\n",
"This notebook investigates a post-processing method for bias mitigation (see figure 21.2 in the book). It based on this [blog](https://www.borealisai.com/research-blogs/tutorial1-bias-and-fairness-ai/) that I wrote for Borealis AI in 2019, which itself was derirved from [this blog](https://research.google.com/bigpicture/attacking-discrimination-in-ml/) by Wattenberg, Viégas, and Hardt.\n",
"This notebook investigates a post-processing method for bias mitigation (see figure 21.2 in the book). It based on this [blog](https://www.borealisai.com/research-blogs/tutorial1-bias-and-fairness-ai/) that I wrote for Borealis AI in 2019, which itself was derived from [this blog](https://research.google.com/bigpicture/attacking-discrimination-in-ml/) by Wattenberg, Viégas, and Hardt.\n",
"\n",
"Work through the cells below, running each cell in turn. In various places you will see the words \"TO DO\". Follow the instructions at these places and make predictions about what is going to happen or write code to complete the functions.\n",
"\n",
@@ -172,7 +172,7 @@
"source": [
"# Blindness to protected attribute\n",
"\n",
"We'll first do the simplest possible thing. We'll choose the same threshold for both blue and yellow populations so that $\\tau_0$ = $\\tau_1$. Basically, we'll ingore what we know about the group membership. Let's see what the ramifications of that."
"We'll first do the simplest possible thing. We'll choose the same threshold for both blue and yellow populations so that $\\tau_0$ = $\\tau_1$. Basically, we'll ignore what we know about the group membership. Let's see what the ramifications of that."
],
"metadata": {
"id": "bE7yPyuWoSUy"
@@ -195,7 +195,7 @@
"source": [
"def compute_probability_get_loan(credit_scores, frequencies, threshold):\n",
" # TODO - Write this function\n",
" # Return the probability that somemone from this group loan based on the frequencies of each\n",
" # Return the probability that someone from this group loan based on the frequencies of each\n",
" # credit score for this group\n",
" # Replace this line:\n",
" prob = 0.5\n",
@@ -297,7 +297,7 @@
"\n",
"This criterion is clearly not great. The blue and yellow groups get given loans at different rates overall, and (for this threshold), the false alarms and true positives are also different, so it's not even fair when we consider whether the loans really were paid back. \n",
"\n",
"TODO -- investigate setting a different threshols $\\tau_{0}=\\tau_{1}$. Is it possible to make the overall rates that loans are given the same? Is it possible to make the false alarm rates the same? Is it possible to make the true positive rates the same?"
"TODO -- investigate setting a different threshold $\\tau_{0}=\\tau_{1}$. Is it possible to make the overall rates that loans are given the same? Is it possible to make the false alarm rates the same? Is it possible to make the true positive rates the same?"
],
"metadata": {
"id": "UCObTsa57uuC"

2
Notebooks/Chap21/21_2_Explainability.ipynb
View File

@@ -400,7 +400,7 @@
{
"cell_type": "markdown",
"source": [
"This model is easilly intepretable. The k'th coeffeicient tells us the how much (and in which direction) changing the value of the k'th input will change the output. This is only valid in the vicinity of the input $x$.\n",
"This model is easily interpretable. The k'th coefficient tells us the how much (and in which direction) changing the value of the k'th input will change the output. This is only valid in the vicinity of the input $x$.\n",
"\n",
"Note that a more sophisticated version of LIME would weight the training points according to how close they are to the original data point of interest."
],