神经网络和相关算法的简单 PyTorch 实现

github地址：

https://github.com/labmlai/annotated_deep_learning_paper_implementations

这是神经网络和相关算法的简单 PyTorch 实现的集合。这些实现与解释一起记录，

该网站 将这些呈现为并排格式化的注释。我们相信这些将帮助您更好地理解这些算法。

我们几乎每周都在积极维护这个 repo 并添加新的实现。 更新。

模块:

✨ Transformers

• Multi-headed attention
• Transformer building blocks
• Transformer XL
  • Relative multi-headed attention
• Rotary Positional Embeddings
• RETRO
• Compressive Transformer
• GPT Architecture
• GLU Variants
• kNN-LM: Generalization through Memorization
• Feedback Transformer
• Switch Transformer
• Fast Weights Transformer
• FNet
• Attention Free Transformer
• Masked Language Model
• MLP-Mixer: An all-MLP Architecture for Vision
• Pay Attention to MLPs (gMLP)
• Vision Transformer (ViT)
• Primer EZ
• Hourglass

✨ Recurrent Highway Networks

✨ LSTM

✨ HyperNetworks – HyperLSTM

✨ ResNet

✨ ConvMixer

✨ Capsule Networks

✨ Generative Adversarial Networks

• Original GAN
• GAN with deep convolutional network
• Cycle GAN
• Wasserstein GAN
• Wasserstein GAN with Gradient Penalty
• StyleGAN 2

✨ Diffusion models

• Denoising Diffusion Probabilistic Models (DDPM)

✨ Sketch RNN

✨ Graph Neural Networks

• Graph Attention Networks (GAT)
• Graph Attention Networks v2 (GATv2)

✨ Counterfactual Regret Minimization (CFR)

Solving games with incomplete information such as poker with CFR.

• Kuhn Poker

✨ Reinforcement Learning

• Proximal Policy Optimization with Generalized Advantage Estimation
• Deep Q Networks with with Dueling Network, Prioritized Replay and Double Q Network.

✨ Optimizers

• Adam
• AMSGrad
• Adam Optimizer with warmup
• Noam Optimizer
• Rectified Adam Optimizer
• AdaBelief Optimizer

✨ Normalization Layers

• Batch Normalization
• Layer Normalization
• Instance Normalization
• Group Normalization
• Weight Standardization
• Batch-Channel Normalization
• DeepNorm

✨ Distillation

✨ Adaptive Computation

• PonderNet

✨ Uncertainty

• Evidential Deep Learning to Quantify Classification Uncertainty

Installation

pip install labml-nn