UvA DL Notebooks

Guides

Guide 1: Working with the Snellius cluster
Guide 2: Research projects with PyTorch
Guide 3: Debugging in PyTorch
Guide 4: Research Projects with JAX

Training Models at Scale

Overview
Part 1.1: Training Larger Models on a Single GPU
Part 1.2: Profiling and Scaling Single-GPU Transformer Models
Part 2.1: Introduction to Distributed Computing in JAX
Part 2.2: (Fully-Sharded) Data Parallelism
Part 3.1: Pipeline Parallelism
Part 3.2: Looping Pipelines
Part 4.1: Tensor Parallelism
Part 4.2: Asynchronous Linear Layers with Tensor Parallelism
Part 4.3: Transformers with Tensor Parallelism
Part 5: Language Modeling with 3D Parallelism

Deep Learning 1 (PyTorch)

Tutorial 2: Introduction to PyTorch
Tutorial 3: Activation Functions
Tutorial 4: Optimization and Initialization
Tutorial 5: Inception, ResNet and DenseNet
Tutorial 6: Transformers and Multi-Head Attention
Tutorial 7: Graph Neural Networks
Tutorial 8: Deep Energy-Based Generative Models
Tutorial 9: Deep Autoencoders
Tutorial 10: Adversarial attacks
Tutorial 11: Normalizing Flows for image modeling
Tutorial 12: Autoregressive Image Modeling
Tutorial 15: Vision Transformers
Tutorial 16: Meta-Learning - Learning to Learn
Tutorial 17: Self-Supervised Contrastive Learning with SimCLR

Deep Learning 1 (JAX+Flax)

Tutorial 2 (JAX): Introduction to JAX+Flax
Tutorial 3 (JAX): Activation Functions
Tutorial 4 (JAX): Optimization and Initialization
Tutorial 5 (JAX): Inception, ResNet and DenseNet
Tutorial 6 (JAX): Transformers and Multi-Head Attention
Tutorial 7 (JAX): Graph Neural Networks
Tutorial 9 (JAX): Deep Autoencoders
Tutorial 11 (JAX): Normalizing Flows for image modeling
Tutorial 12 (JAX): Autoregressive Image Modeling
Tutorial 15 (JAX): Vision Transformers
Tutorial 17 (JAX): Self-Supervised Contrastive Learning with SimCLR

Deep Learning 2

GDL - Regular Group Convolutions
GDL - Steerable CNNs
DPM1 - Deep Probabilistic Models I
DPM2 - Variational inference for deep discrete latent variable models
DPM 2 - Variational Inference for Deep Continuous LVMs
AGM - Advanced Topics in Normalizing Flows - 1x1 convolution
HDL - Introduction to HyperParameter Tuning
HDL - Introduction to Multi GPU Programming
Tutorial 1: Bayesian Neural Networks with Pyro
Tutorial 2: Comparison to other methods of uncertainty quantification
DNN - Tutorial 2 Part I: Physics inspired Machine Learning
DNN - Tutorial 2 Part II: Physics inspired Machine Learning
DS - Dynamical Systems & Neural ODEs
SGA - Sampling Discrete Structures
SGA - Sampling Subsets with Gumbel-Top \(k\) Relaxations
SGA: Learning Latent Permutations with Gumbel-Sinkhorn Networks
SGA - Graph Sampling for Neural Relational Inference
CRL - Causal Identifiability from Temporal Intervened Sequences

Work in Progress Pure PyTorch

Work in Progress for the new PyTorch version

UvA DL Notebooks

Work in Progress for the new PyTorch version
View page source

Work in Progress for the new PyTorch version

By: Samuele Papa, Alex Gabel.

[1]:

import torch

[2]:

t = torch.ones(16,16)

[3]:

t.shape

[3]:

torch.Size([16, 16])

Previous

© Copyright 2022, Phillip Lippe.

Built with Sphinx using a theme provided by Read the Docs.