383 lines
24 KiB
HTML
383 lines
24 KiB
HTML
<!DOCTYPE html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="UTF-8">
|
|
<title>udlbook</title>
|
|
<link rel="stylesheet" href="style.css">
|
|
</head>
|
|
|
|
<body>
|
|
<div id="head">
|
|
<div>
|
|
<h1 style="margin: 0; font-size: 36px">Understanding Deep Learning</h1>
|
|
by Simon J.D. Prince
|
|
<br>To be published by MIT Press Dec 5th 2023.<br>
|
|
<ul>
|
|
<li>
|
|
<p style="font-size: larger; margin-bottom: 0">Download draft PDF Chapters 1-21 <a
|
|
href="https://github.com/udlbook/udlbook/releases/download/v1.14/UnderstandingDeepLearning_13_10_23_C.pdf">here</a>
|
|
</p>2023-10-13. CC-BY-NC-ND license<br>
|
|
<img src="https://img.shields.io/github/downloads/udlbook/udlbook/total" alt="download stats shield">
|
|
</li>
|
|
<li> Report errata via <a href="https://github.com/udlbook/udlbook/issues">github</a>
|
|
or contact me directly at udlbookmail@gmail.com
|
|
<li> Follow me on <a href="https://twitter.com/SimonPrinceAI">Twitter</a> or <a
|
|
href="https://www.linkedin.com/in/simon-prince-615bb9165/">LinkedIn</a> for updates.
|
|
</ul>
|
|
<h2>Table of contents</h2>
|
|
<ul>
|
|
<li> Chapter 1 - Introduction
|
|
<li> Chapter 2 - Supervised learning
|
|
<li> Chapter 3 - Shallow neural networks
|
|
<li> Chapter 4 - Deep neural networks
|
|
<li> Chapter 5 - Loss functions
|
|
<li> Chapter 6 - Training models
|
|
<li> Chapter 7 - Gradients and initialization
|
|
<li> Chapter 8 - Measuring performance
|
|
<li> Chapter 9 - Regularization
|
|
<li> Chapter 10 - Convolutional networks
|
|
<li> Chapter 11 - Residual networks
|
|
<li> Chapter 12 - Transformers
|
|
<li> Chapter 13 - Graph neural networks
|
|
<li> Chapter 14 - Unsupervised learning
|
|
<li> Chapter 15 - Generative adversarial networks
|
|
<li> Chapter 16 - Normalizing flows
|
|
<li> Chapter 17 - Variational autoencoders
|
|
<li> Chapter 18 - Diffusion models
|
|
<li> Chapter 19 - Deep reinforcement learning
|
|
<li> Chapter 20 - Why does deep learning work?
|
|
<li> Chapter 21 - Deep learning and ethics
|
|
</ul>
|
|
</div>
|
|
<div id="cover">
|
|
<img src="https://raw.githubusercontent.com/udlbook/udlbook/main/UDLCoverSmall.jpg"
|
|
alt="front cover">
|
|
</div>
|
|
</div>
|
|
<div id="body">
|
|
<h2>Resources for instructors </h2>
|
|
<p>Instructor answer booklet available with proof of credentials via <a
|
|
href="https://mitpress.mit.edu/9780262048644/understanding-deep-learning"> MIT Press</a>.</p>
|
|
<p>Figures in PDF (vector) / SVG (vector) / Powerpoint (images):
|
|
<ul>
|
|
<li> Chapter 1 - Introduction: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap1PDF.zip">PDF
|
|
Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1udnl5pUOAc8DcAQ7HQwyzP9pwL95ynnv">
|
|
SVG
|
|
Figures</a> / <a
|
|
href="https://docs.google.com/presentation/d/1IjTqIUvWCJc71b5vEJYte-Dwujcp7rvG/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
|
|
Figures</a>
|
|
<li> Chapter 2 - Supervised learning: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap2PDF.zip">PDF Figures</a> / <a
|
|
href="https://drive.google.com/uc?export=download&id=1VSxcU5y1qNFlmd3Lb3uOWyzILuOj1Dla"> SVG Figures</a>
|
|
/
|
|
<a href="https://docs.google.com/presentation/d/1Br7R01ROtRWPlNhC_KOommeHAWMBpWtz/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
|
|
Figures</a>
|
|
<li> Chapter 3 - Shallow neural networks: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap3PDF.zip">PDF Figures</a> / <a
|
|
href="https://drive.google.com/uc?export=download&id=19kZFWlXhzN82Zx02ByMmSZOO4T41fmqI"> SVG Figures</a>
|
|
/
|
|
<a href="https://docs.google.com/presentation/d/1e9M3jB5I9qZ4dCBY90Q3Hwft_i068QVQ/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
|
|
Figures</a>
|
|
<li> Chapter 4 - Deep neural networks: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap4PDF.zip">PDF Figures</a> / <a
|
|
href="https://drive.google.com/uc?export=download&id=1ojr0ebsOhzvS04ItAflX2cVmYqHQHZUa"> SVG Figures</a>
|
|
/
|
|
<a href="https://docs.google.com/presentation/d/1LTSsmY4mMrJbqXVvoTOCkQwHrRKoYnJj/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
|
|
Figures</a>
|
|
<li> Chapter 5 - Loss functions: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap5PDF.zip">PDF
|
|
Figures</a> / <a href="https://drive.google.com/uc?export=download&id=17MJO7fiMpFZVqKeqXTbQ36AMpmR4GizZ">
|
|
SVG
|
|
Figures</a> / <a
|
|
href="https://docs.google.com/presentation/d/1gcpC_3z9oRp87eMkoco-kdLD-MM54Puk/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
|
|
Figures</a>
|
|
<li> Chapter 6 - Training models: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap6PDF.zip">PDF
|
|
Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1VPdhFRnCr9_idTrX0UdHKGAw2shUuwhK">
|
|
SVG
|
|
Figures</a> / <a
|
|
href="https://docs.google.com/presentation/d/1AKoeggAFBl9yLC7X5tushAGzCCxmB7EY/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
|
|
Figures</a>
|
|
<li> Chapter 7 - Gradients and initialization: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap7PDF.zip">PDF Figures</a> / <a
|
|
href="https://drive.google.com/uc?export=download&id=1TTl4gvrTvNbegnml4CoGoKOOd6O8-PGs"> SVG Figures</a>
|
|
/
|
|
<a href="https://docs.google.com/presentation/d/11zhB6PI-Dp6Ogmr4IcI6fbvbqNqLyYcz/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
|
|
Figures</a>
|
|
<li> Chapter 8 - Measuring performance: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap8PDF.zip">PDF Figures</a> / <a
|
|
href="https://drive.google.com/uc?export=download&id=19eQOnygd_l0DzgtJxXuYnWa4z7QKJrJx"> SVG Figures</a>
|
|
/
|
|
<a href="https://docs.google.com/presentation/d/1SHRmJscDLUuQrG7tmysnScb3ZUAqVMZo/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
|
|
Figures</a>
|
|
<li> Chapter 9 - Regularization: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap9PDF.zip">PDF
|
|
Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1LprgnUGL7xAM9-jlGZC9LhMPeefjY0r0">
|
|
SVG
|
|
Figures</a> / <a
|
|
href="https://docs.google.com/presentation/d/1VwIfvjpdfTny6sEfu4ZETwCnw6m8Eg-5/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
|
|
Figures</a>
|
|
<li> Chapter 10 - Convolutional networks: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap10PDF.zip">PDF Figures</a> / <a
|
|
href="https://drive.google.com/uc?export=download&id=1-Wb3VzaSvVeRzoUzJbI2JjZE0uwqupM9"> SVG Figures</a>
|
|
/
|
|
<a href="https://docs.google.com/presentation/d/1MtfKBC4Y9hWwGqeP6DVwUNbi1j5ncQCg/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
|
|
Figures</a>
|
|
<li> Chapter 11 - Residual networks: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap11PDF.zip">PDF Figures</a> / <a
|
|
href="https://drive.google.com/uc?export=download&id=1Mr58jzEVseUAfNYbGWCQyDtEDwvfHRi1"> SVG Figures</a>
|
|
/
|
|
<a href="https://docs.google.com/presentation/d/1saY8Faz0KTKAAifUrbkQdLA2qkyEjOPI/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
|
|
Figures</a>
|
|
<li> Chapter 12 - Transformers: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap12PDF.zip">PDF
|
|
Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1txzOVNf8-jH4UfJ6SLnrtOfPd1Q3ebzd">
|
|
SVG
|
|
Figures</a> / <a
|
|
href="https://docs.google.com/presentation/d/1GVNvYWa0WJA6oKg89qZre-UZEhABfm0l/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
|
|
Figures</a>
|
|
<li> Chapter 13 - Graph neural networks: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap13PDF.zip">PDF Figures</a> / <a
|
|
href="https://drive.google.com/uc?export=download&id=1lQIV6nRp6LVfaMgpGFhuwEXG-lTEaAwe"> SVG Figures</a>
|
|
/
|
|
<a href="https://docs.google.com/presentation/d/1YwF3U82c1mQ74c1WqHVTzLZ0j7GgKaWP/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
|
|
Figures</a>
|
|
<li> Chapter 14 - Unsupervised learning: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap14PDF.zip">PDF Figures</a> / <a
|
|
href="https://drive.google.com/uc?export=download&id=1aMbI6iCuUvOywqk5pBOmppJu1L1anqsM"> SVG Figures</a>
|
|
/
|
|
<a href="https://docs.google.com/presentation/d/1A-lBGv3NHl4L32NvfFgy1EKeSwY-0UeB/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">
|
|
PowerPoint Figures</a>
|
|
<li> Chapter 15 - Generative adversarial networks: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap15PDF.zip">PDF Figures</a> / <a
|
|
href="https://drive.google.com/uc?export=download&id=1EErnlZCOlXc3HK7m83T2Jh_0NzIUHvtL"> SVG Figures</a>
|
|
/
|
|
<a href="https://docs.google.com/presentation/d/10Ernk41ShOTf4IYkMD-l4dJfKATkXH4w/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
|
|
Figures</a>
|
|
<li> Chapter 16 - Normalizing flows: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap16PDF.zip">PDF Figures</a> / <a
|
|
href="https://drive.google.com/uc?export=download&id=1B9bxtmdugwtg-b7Y4AdQKAIEVWxjx8l3"> SVG Figures</a>
|
|
/
|
|
<a href="https://docs.google.com/presentation/d/1nLLzqb9pdfF_h6i1HUDSyp7kSMIkSUUA/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
|
|
Figures</a>
|
|
<li> Chapter 17 - Variational autoencoders: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap17PDF.zip">PDF Figures</a> / <a
|
|
href="https://drive.google.com/uc?export=download&id=1SNtNIY7khlHQYMtaOH-FosSH3kWwL4b7"> SVG Figures</a>
|
|
/
|
|
<a href="https://docs.google.com/presentation/d/1lQE4Bu7-LgvV2VlJOt_4dQT-kusYl7Vo/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
|
|
Figures</a>
|
|
<li> Chapter 18 - Diffusion models: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap18PDF.zip">PDF Figures</a> / <a
|
|
href="https://docs.google.com/presentation/d/1x_ufIBtVPzWUvRieKMkpw5SdRjXWwdfR/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">
|
|
PowerPoint Figures</a>
|
|
<li> Chapter 19 - Deep reinforcement learning: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap19PDF.zip">PDF Figures</a> / <a
|
|
href="https://drive.google.com/uc?export=download&id=1a5WUoF7jeSgwC_PVdckJi1Gny46fCqh0"> SVG Figures</a>
|
|
/
|
|
<a href="https://docs.google.com/presentation/d/1TnYmVbFNhmMFetbjyfXGmkxp1EHauMqr/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">
|
|
PowerPoint Figures </a>
|
|
<li> Chapter 20 - Why does deep learning work?: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap20PDF.zip">PDF Figures</a> / <a
|
|
href="https://drive.google.com/uc?export=download&id=1M2d0DHEgddAQoIedKSDTTt7m1ZdmBLQ3"> SVG Figures</a>
|
|
/
|
|
<a href="https://docs.google.com/presentation/d/1coxF4IsrCzDTLrNjRagHvqB_FBy10miA/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">
|
|
PowerPoint Figures</a>
|
|
<li> Chapter 21 - Deep learning and ethics: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap21PDF.zip">PDF Figures</a> / <a
|
|
href="https://drive.google.com/uc?export=download&id=1jixmFfwmZkW_UVYzcxmDcMsdFFtnZ0bU"> SVG Figures</a>/
|
|
<a
|
|
href="https://docs.google.com/presentation/d/1EtfzanZYILvi9_-Idm28zD94I_6OrN9R/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint
|
|
Figures</a>
|
|
<li> Appendices - <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLAppendixPDF.zip">PDF
|
|
Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1k2j7hMN40ISPSg9skFYWFL3oZT7r8v-l">
|
|
SVG
|
|
Figures</a> / <a
|
|
href="https://docs.google.com/presentation/d/1_2cJHRnsoQQHst0rwZssv-XH4o5SEHks/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">Powerpoint
|
|
Figures</a>
|
|
</ul>
|
|
|
|
Instructions for editing figures / equations can be found <a
|
|
href="https://drive.google.com/file/d/1T_MXXVR4AfyMnlEFI-UVDh--FXI5deAp/view?usp=sharing">here</a>.
|
|
|
|
<h2>Resources for students</h2>
|
|
|
|
<p>Answers to selected questions: <a
|
|
href="https://github.com/udlbook/udlbook/raw/main/UDL_Answer_Booklet_Students.pdf">PDF</a>
|
|
</p>
|
|
<p>Python notebooks: (Early ones more thoroughly tested than later ones!)</p>
|
|
|
|
<ul>
|
|
<li> Notebook 1.1 - Background mathematics: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap01/1_1_BackgroundMathematics.ipynb">ipynb/colab</a>
|
|
</li>
|
|
<li> Notebook 2.1 - Supervised learning: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap02/2_1_Supervised_Learning.ipynb">ipynb/colab</a>
|
|
</li>
|
|
<li> Notebook 3.1 - Shallow networks I: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_1_Shallow_Networks_I.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 3.2 - Shallow networks II: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_2_Shallow_Networks_II.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 3.3 - Shallow network regions: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_3_Shallow_Network_Regions.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 3.4 - Activation functions: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_4_Activation_Functions.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 4.1 - Composing networks: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap04/4_1_Composing_Networks.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 4.2 - Clipping functions: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap04/4_2_Clipping_functions.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 4.3 - Deep networks: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap04/4_3_Deep_Networks.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 5.1 - Least squares loss: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap05/5_1_Least_Squares_Loss.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 5.2 - Binary cross-entropy loss: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap05/5_2_Binary_Cross_Entropy_Loss.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 5.3 - Multiclass cross-entropy loss: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap05/5_3_Multiclass_Cross_entropy_Loss.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 6.1 - Line search: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_1_Line_Search.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 6.2 - Gradient descent: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_2_Gradient_Descent.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 6.3 - Stochastic gradient descent: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_3_Stochastic_Gradient_Descent.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 6.4 - Momentum: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_4_Momentum.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 6.5 - Adam: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_5_Adam.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 7.1 - Backpropagation in toy model: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap07/7_1_Backpropagation_in_Toy_Model.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 7.2 - Backpropagation: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap07/7_2_Backpropagation.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 7.3 - Initialization: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap07/7_3_Initialization.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 8.1 - MNIST-1D performance: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap08/8_1_MNIST_1D_Performance.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 8.2 - Bias-variance trade-off: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap08/8_2_Bias_Variance_Trade_Off.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 8.3 - Double descent: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap08/8_3_Double_Descent.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 8.4 - High-dimensional spaces: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap08/8_4_High_Dimensional_Spaces.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 9.1 - L2 regularization: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap09/9_1_L2_Regularization.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 9.2 - Implicit regularization: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap09/9_2_Implicit_Regularization.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 9.3 - Ensembling: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap09/9_3_Ensembling.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 9.4 - Bayesian approach: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap09/9_4_Bayesian_Approach.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 9.5 - Augmentation <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap09/9_5_Augmentation.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 10.1 - 1D convolution: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap10/10_1_1D_Convolution.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 10.2 - Convolution for MNIST-1D: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap10/10_2_Convolution_for_MNIST_1D.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 10.3 - 2D convolution: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap10/10_3_2D_Convolution.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 10.4 - Downsampling & upsampling: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap10/10_4_Downsampling_and_Upsampling.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 10.5 - Convolution for MNIST: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap10/10_5_Convolution_For_MNIST.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 11.1 - Shattered gradients: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap11/11_1_Shattered_Gradients.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 11.2 - Residual networks: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap11/11_2_Residual_Networks.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 11.3 - Batch normalization: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap11/11_3_Batch_Normalization.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 12.1 - Self-attention: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap12/12_1_Self_Attention.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 12.2 - Multi-head self-attention: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap12/12_2_Multihead_Self_Attention.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 12.3 - Tokenization: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap12/12_3_Tokenization.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 12.4 - Decoding strategies: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap12/12_4_Decoding_Strategies.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 13.1 - Encoding graphs: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap13/13_1_Graph_Representation.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 13.2 - Graph classification : <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap13/13_2_Graph_Classification.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 13.3 - Neighborhood sampling: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap13/13_3_Neighborhood_Sampling.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 13.4 - Graph attention: <a
|
|
href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap13/13_4_Graph_Attention_Networks.ipynb">ipynb/colab </a>
|
|
</li>
|
|
<li> Notebook 15.1 - GAN toy example: (coming soon)</li>
|
|
<li> Notebook 15.2 - Wasserstein distance: (coming soon)</li>
|
|
<li> Notebook 16.1 - 1D normalizing flows: (coming soon)</li>
|
|
<li> Notebook 16.2 - Autoregressive flows: (coming soon)</li>
|
|
<li> Notebook 16.3 - Contraction mappings: (coming soon)</li>
|
|
<li> Notebook 17.1 - Latent variable models: (coming soon)</li>
|
|
<li> Notebook 17.2 - Reparameterization trick: (coming soon)</li>
|
|
<li> Notebook 17.3 - Importance sampling: (coming soon)</li>
|
|
<li> Notebook 18.1 - Diffusion encoder: (coming soon)</li>
|
|
<li> Notebook 18.2 - 1D diffusion model: (coming soon)</li>
|
|
<li> Notebook 18.3 - Reparameterized model: (coming soon)</li>
|
|
<li> Notebook 18.4 - Families of diffusion models: (coming soon)</li>
|
|
<li> Notebook 19.1 - Markov decision processes: (coming soon)</li>
|
|
<li> Notebook 19.2 - Dynamic programming: (coming soon)</li>
|
|
<li> Notebook 19.3 - Monte-Carlo methods: (coming soon)</li>
|
|
<li> Notebook 19.4 - Temporal difference methods: (coming soon)</li>
|
|
<li> Notebook 19.5 - Control variates: (coming soon)</li>
|
|
<li> Notebook 20.1 - Random data: (coming soon)</li>
|
|
<li> Notebook 20.2 - Full-batch gradient descent: (coming soon)</li>
|
|
<li> Notebook 20.3 - Lottery tickets: (coming soon)</li>
|
|
<li> Notebook 20.4 - Adversarial attacks: (coming soon)</li>
|
|
<li> Notebook 21.1 - Bias mitigation: (coming soon)</li>
|
|
<li> Notebook 21.2 - Explainability: (coming soon)</li>
|
|
</ul>
|
|
|
|
|
|
<br>
|
|
<h2>Citation</h2>
|
|
<pre><code>
|
|
@book{prince2023understanding,
|
|
author = "Simon J.D. Prince",
|
|
title = "Understanding Deep Learning",
|
|
publisher = "MIT Press",
|
|
year = 2023,
|
|
url = "http://udlbook.com"
|
|
}
|
|
</code></pre>
|
|
</div>
|
|
</body> |