Step 1: Generative Deep Learning Foundation

Let’s start with the first step.

Step 1: Generative Deep Learning: Teaching Machines to Paint, Write, Compose, and Play 2nd

One of the best resources to get started with generative AI is the book "Generative Deep Learning" by David Foster. This book is an excellent introduction to the field of generative models.

It covers practical implementations of several popular generative models, including:

  1. VAEs (Variational Autoencoders),

  2. GAN (Generative Adversarial Network),

  3. Autoregressive Models

  4. An introduction to Transformers and attention mechanisms

While the book primarily focuses on the image domain, the concepts and techniques introduced are foundational and transferable to other data types, especially text.

It gives a good balance between intuition, practical coding, and light mathematical theory, making it approachable even for those without a strong math background. The book uses Tensorflow as its main framework for the hands-on. However, I suggest using PyTorch for the subsequent guides, as it's widely adopted and beginner-friendly. There's good news, a PyTorch implementation of the book is available in this GitHub repository.

Here’s the outline of the book:

  • Part I: Introduction to Generative Deep Learning: Ch 1 on Generative Modeling, Ch 2 on Deep Learning

  • Part II: Methods: Ch 3 on Variational Autoencoders, Ch 4 on Generative Adversarial Networks, Ch 5 on Autoregressive Models, Ch 6 Normalizing Flows, Ch 7 Energy-Based Models, Ch 8 Diffusion Models

  • Part III: Applications: Ch 9 Transformers: Ch 10 on Advanced GANs, Ch 11 on Music Generation, Ch 12 on World Models, Ch 13 on Multimodal Models, Ch 14 on Conclusion

If you feel overwhelmed by the number of chapters, here’s a recommended path to get the most essential ideas:

  • Finish Part I to build your foundation

  • Do Chapters 3–5 from Part II to learn the core generative techniques

  • Jump to Chapter 9 from Part III to understand Transformers

You can always revisit the rest of the chapters later once you're more comfortable.

After this, you're ready to move on to Step 2.

PreviousHow to Get Started with Generative Models and Large Language Models (LLMs)NextStep 2: Large Language Model (from scratch)

Last updated