Transformer

Annotated Transformer, in PyTorch http://nlp.seas.harvard.edu/annotated-transformer/

Illustrated GPT-2 https://jalammar.github.io/illustrated-gpt2/

Domain specific architectures for AI inference

GPU Book https://huggingface.co/spaces/nanotron/ultrascale-playbook

TPU Book https://jax-ml.github.io/scaling-book

大模型分布式训练并行技术（一）-概述

大模型分布式训练并行技术（四）-张量并行

https://www.chipstrat.com/p/gpu-networking-basics-part-1

https://unsloth.ai/