119 lines
14 KiB
HTML
119 lines
14 KiB
HTML
<h1>Understanding Deep Learning</h1>
|
|
by Simon J.D. Prince
|
|
<br>
|
|
To be published by MIT Press Dec 5th 2023.<br>
|
|
|
|
<img src="https://raw.githubusercontent.com/udlbook/udlbook/main/UDLCoverSmall.jpg" alt="front cover">
|
|
|
|
<h2> Download draft PDF </h2>
|
|
|
|
<a href="https://github.com/udlbook/udlbook/releases/download/v1.1/UnderstandingDeepLearning_23_07_23_C.pdf">Draft PDF Chapters 1-21</a><br> 2023-07-23. CC-BY-NC-ND license
|
|
<br>
|
|
<img src="https://img.shields.io/github/downloads/udlbook/udlbook/total" alt="download stats shield">
|
|
<br>
|
|
<ul>
|
|
<li> Appendices and notebooks coming soon
|
|
<li> Report errata via <a href="https://github.com/udlbook/udlbook/issues">github</a> or contact me directly at udlbookmail@gmail.com
|
|
<li> Follow me on <a href="https://twitter.com/SimonPrinceAI">Twitter</a> or <a href="https://www.linkedin.com/in/simon-prince-615bb9165/">LinkedIn</a> for updates.
|
|
</ul>
|
|
|
|
<h2>Table of contents</h2>
|
|
<ul>
|
|
<li> Chapter 1 - Introduction
|
|
<li> Chapter 2 - Supervised learning
|
|
<li> Chapter 3 - Shallow neural networks
|
|
<li> Chapter 4 - Deep neural networks
|
|
<li> Chapter 5 - Loss functions
|
|
<li> Chapter 6 - Training models
|
|
<li> Chapter 7 - Gradients and initialization
|
|
<li> Chapter 8 - Measuring performance
|
|
<li> Chapter 9 - Regularization
|
|
<li> Chapter 10 - Convolutional networks
|
|
<li> Chapter 11 - Residual networks
|
|
<li> Chapter 12 - Transformers
|
|
<li> Chapter 13 - Graph neural networks
|
|
<li> Chapter 14 - Unsupervised learning
|
|
<li> Chapter 15 - Generative adversarial networks
|
|
<li> Chapter 16 - Normalizing flows
|
|
<li> Chapter 17 - Variational autoencoders
|
|
<li> Chapter 18 - Diffusion models
|
|
<li> Chapter 19 - Deep reinforcement learning
|
|
<li> Chapter 20 - Why does deep learning work?
|
|
<li> Chapter 21 - Deep learning and ethics
|
|
</ul>
|
|
|
|
|
|
<h2>Resources for instructors </h2>
|
|
|
|
<p></p>Instructor answer booklet available with proof of credentials via <a href="https://mitpress.mit.edu/9780262048644/understanding-deep-learning"/> MIT Press</a></p>
|
|
|
|
<p></p>Figures in PDF (vector) / SVG (vector) / Powerpoint (images):
|
|
|
|
<ul>
|
|
<li> Chapter 1 - Introduction: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap1PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1udnl5pUOAc8DcAQ7HQwyzP9pwL95ynnv"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1IjTqIUvWCJc71b5vEJYte-Dwujcp7rvG/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
|
|
<li> Chapter 2 - Supervised learning: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap2PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1VSxcU5y1qNFlmd3Lb3uOWyzILuOj1Dla"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1Br7R01ROtRWPlNhC_KOommeHAWMBpWtz/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
|
|
<li> Chapter 3 - Shallow neural networks: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap3PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=19kZFWlXhzN82Zx02ByMmSZOO4T41fmqI"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1e9M3jB5I9qZ4dCBY90Q3Hwft_i068QVQ/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
|
|
<li> Chapter 4 - Deep neural networks: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap4PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1ojr0ebsOhzvS04ItAflX2cVmYqHQHZUa"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1LTSsmY4mMrJbqXVvoTOCkQwHrRKoYnJj/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
|
|
<li> Chapter 5 - Loss functions: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap5PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=17MJO7fiMpFZVqKeqXTbQ36AMpmR4GizZ"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1gcpC_3z9oRp87eMkoco-kdLD-MM54Puk/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
|
|
<li> Chapter 6 - Training models: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap6PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1VPdhFRnCr9_idTrX0UdHKGAw2shUuwhK"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1AKoeggAFBl9yLC7X5tushAGzCCxmB7EY/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
|
|
<li> Chapter 7 - Gradients and initialization: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap7PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1TTl4gvrTvNbegnml4CoGoKOOd6O8-PGs"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/11zhB6PI-Dp6Ogmr4IcI6fbvbqNqLyYcz/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
|
|
<li> Chapter 8 - Measuring performance: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap8PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=19eQOnygd_l0DzgtJxXuYnWa4z7QKJrJx"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1SHRmJscDLUuQrG7tmysnScb3ZUAqVMZo/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
|
|
<li> Chapter 9 - Regularization: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap9PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1LprgnUGL7xAM9-jlGZC9LhMPeefjY0r0"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1VwIfvjpdfTny6sEfu4ZETwCnw6m8Eg-5/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
|
|
<li> Chapter 10 - Convolutional networks: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap10PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1-Wb3VzaSvVeRzoUzJbI2JjZE0uwqupM9"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1MtfKBC4Y9hWwGqeP6DVwUNbi1j5ncQCg/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
|
|
<li> Chapter 11 - Residual networks: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap11PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1Mr58jzEVseUAfNYbGWCQyDtEDwvfHRi1"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1saY8Faz0KTKAAifUrbkQdLA2qkyEjOPI/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
|
|
<li> Chapter 12 - Transformers: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap12PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1txzOVNf8-jH4UfJ6SLnrtOfPd1Q3ebzd"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1GVNvYWa0WJA6oKg89qZre-UZEhABfm0l/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
|
|
<li> Chapter 13 - Graph neural networks: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap13PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1lQIV6nRp6LVfaMgpGFhuwEXG-lTEaAwe"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1YwF3U82c1mQ74c1WqHVTzLZ0j7GgKaWP/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
|
|
<li> Chapter 14 - Unsupervised learning: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap14PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1aMbI6iCuUvOywqk5pBOmppJu1L1anqsM"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1A-lBGv3NHl4L32NvfFgy1EKeSwY-0UeB/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true"> Powerpoint Figures</a>
|
|
<li> Chapter 15 - Generative adversarial networks: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap15PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1EErnlZCOlXc3HK7m83T2Jh_0NzIUHvtL"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/10Ernk41ShOTf4IYkMD-l4dJfKATkXH4w/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
|
|
<li> Chapter 16 - Normalizing flows: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap16PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1B9bxtmdugwtg-b7Y4AdQKAIEVWxjx8l3"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1nLLzqb9pdfF_h6i1HUDSyp7kSMIkSUUA/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
|
|
<li> Chapter 17 - Variational autoencoders: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap17PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1SNtNIY7khlHQYMtaOH-FosSH3kWwL4b7"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1lQE4Bu7-LgvV2VlJOt_4dQT-kusYl7Vo/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
|
|
<li> Chapter 18 - Diffusion models: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap18PDF.zip">PDF Figures</a> / <a href="https://docs.google.com/presentation/d/1x_ufIBtVPzWUvRieKMkpw5SdRjXWwdfR/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true"> PowerPoint Figures</a>
|
|
<li> Chapter 19 - Deep reinforcement learning: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap19PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1a5WUoF7jeSgwC_PVdckJi1Gny46fCqh0"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1TnYmVbFNhmMFetbjyfXGmkxp1EHauMqr/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true"> PowerPoint Figures </a>
|
|
<li> Chapter 20 - Why does deep learning work?: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap20PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1M2d0DHEgddAQoIedKSDTTt7m1ZdmBLQ3"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1coxF4IsrCzDTLrNjRagHvqB_FBy10miA/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true"> PowerPoint Figures</a>
|
|
<li> Chapter 21 - Deep learning and ethics: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap21PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1jixmFfwmZkW_UVYzcxmDcMsdFFtnZ0bU"> SVG Figures</a>/ <a href="https://docs.google.com/presentation/d/1EtfzanZYILvi9_-Idm28zD94I_6OrN9R/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
|
|
<li> Appendices - <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLAppendixPDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1k2j7hMN40ISPSg9skFYWFL3oZT7r8v-l"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1_2cJHRnsoQQHst0rwZssv-XH4o5SEHks/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">Powerpoint Figures</a>
|
|
</ul>
|
|
|
|
Instructions for editing figures / equations can be found <a href="https://drive.google.com/uc?export=download&id=1T_MXXVR4AfyMnlEFI-UVDh--FXI5deAp/">here</a>.</p>
|
|
|
|
<h2>Resources for students</h2>
|
|
|
|
<p>Answers to selected questions: <a href="https://github.com/udlbook/udlbook/raw/main/UDL_Answer_Booklet_Students.pdf">PDF</a></p>
|
|
<p>Python notebooks:</p>
|
|
|
|
<ul>
|
|
<li> Chapter 1 - Introduction: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap01/1_1_BackgroundMathematics.ipynb">1.1</a>
|
|
<li> Chapter 2 - Supervised learning <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap02/2_1_Supervised_Learning.ipynb">2.1</a>
|
|
<li> Chapter 3 - Shallow neural networks <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_1_Shallow_Networks_I.ipynb">3.1</a>,<a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_2_Shallow_Networks_II.ipynb">3.2</a>,<a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_3_Shallow_Network_Regions.ipynb">3.3</a>,<a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_4_Activation_Functions.ipynb">3.4</a>
|
|
<li> Chapter 4 - Deep neural networks <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap04/4_1_Composing_Networks.ipynb">4.1</a>, 4.2, 4.3 (coming soon)
|
|
<li> Chapter 5 - Loss functions (coming soon)
|
|
<li> Chapter 6 - Training models (coming soon)
|
|
<li> Chapter 7 - Gradients and initialization (coming soon)
|
|
<li> Chapter 8 - Measuring performance (coming soon)
|
|
<li> Chapter 9 - Regularization (coming soon)
|
|
<li> Chapter 10 - Convolutional networks (coming soon)
|
|
<li> Chapter 11 - Residual networks (coming soon)
|
|
<li> Chapter 12 - Transformers (coming soon)
|
|
<li> Chapter 13 - Graph neural networks (coming soon)
|
|
<li> Chapter 14 - Unsupervised learning (coming soon)
|
|
<li> Chapter 15 - Generative adversarial networks (coming soon)
|
|
<li> Chapter 16 - Normalizing flows (coming soon)
|
|
<li> Chapter 17 - Variational autoencoders (coming soon)
|
|
<li> Chapter 18 - Diffusion models (coming soon)
|
|
<li> Chapter 19 - Deep reinforcement learning (coming soon)
|
|
<li> Chapter 20 - Why does deep learning work? (coming soon)
|
|
<li> Chapter 21 - Deep learning and ethics (coming soon)
|
|
</ul>
|
|
|
|
<br>
|
|
<h2>Citation:</h2>
|
|
<pre><code>
|
|
@book{prince2023understanding,
|
|
author = "Simon J.D. Prince",
|
|
title = "Understanding Deep Learning",
|
|
publisher = "MIT Press",
|
|
year = 2023,
|
|
url = "http://udlbook.com"
|
|
}
|
|
</code></pre>
|