← Back to Tutorials

12. References & Resources

References

Generative AI Cheatsheet

Model	Type	Key Innovation	Year
VAE	Autoencoder	Probabilistic latent space + KL regularization	2013
GAN	Adversarial	Generator + Discriminator minimax game	2014
Transformer	Attention	Self-attention replaces recurrence	2017
GPT-1/2/3/4	Autoregressive	Decoder-only transformer, scaling laws	2018-2023
StyleGAN	GAN	Style-based generator, disentangled features	2019
DDPM	Diffusion	Denoising diffusion probabilistic models	2020
Stable Diffusion	Diffusion	Latent diffusion, open-source, fast	2022
ChatGPT	Autoregressive	RLHF alignment, conversational interface	2022

Glossary

Attention	Mechanism that weighs the importance of different input elements
Autoregressive	Generating one token at a time conditioned on previous tokens
Diffusion	Gradual noising → reverse denoising process
Discriminator	Network that distinguishes real from generated data (in GANs)
Embedding	Dense vector representation of a token or feature
Generator	Network that creates synthetic data (in GANs)
Hallucination	Model generating plausible but incorrect information
Latent Space	Low-dimensional representation learned by the model
LoRA	Low-Rank Adaptation — efficient fine-tuning technique
MLE	Maximum Likelihood Estimation — finding parameters that maximize data probability
Mode Collapse	GAN generator producing limited output varieties
Positional Encoding	Information added to embeddings to indicate token position
RLHF	Reinforcement Learning from Human Feedback
Softmax	Normalizes logits into a probability distribution

Useful Resources

Hugging Face — Models, datasets, spaces, course
OpenAI Cookbook — API examples and guides
Papers with Code — Research papers + implementations
The AI Summer — Tutorials and explanations
Lil'Log — In-depth blog on generative models
arXiv — Research preprints (search: cs.LG, cs.CL, cs.CV)

Key Research Papers

"Auto-Encoding Variational Bayes" — Kingma & Welling (2013)
"Generative Adversarial Nets" — Goodfellow et al. (2014)
"Attention Is All You Need" — Vaswani et al. (2017)
"Denoising Diffusion Probabilistic Models" — Ho et al. (2020)
"Training Language Models to Follow Instructions with Human Feedback" — Ouyang et al. (2022)

Congratulations! You have completed the Generative AI Tutorial. You now have a comprehensive understanding of how generative models work, from GANs and VAEs to transformers and diffusion models. Keep experimenting and building!