Essentials of Embodied AI Research¶

A structured learning resource for PhD students

From reinforcement learning foundations to embodied intelligence systems

Welcome! This is a comprehensive, open-source educational resource designed to help early-stage PhD students build a solid foundation in Embodied AI (EAI) research. The content spans reinforcement learning theory, world models, embodied systems, and the distributed infrastructure that ties it all together.

Who Is This For?¶

This resource is designed for:

Early-stage PhD students entering RL, robotics, or embodied AI research
Engineers transitioning from ML/DL into embodied systems
Researchers looking for a structured reference across these interconnected fields

We assume familiarity with:

Linear algebra, probability, and calculus
Basic machine learning and deep learning (CNNs, Transformers)
Python programming and PyTorch/JAX

What's Covered¶

Part I: Reinforcement Learning¶

Core RL theory — MDPs, Bellman equations, policy gradients, value-based methods, actor-critic, model-based RL, and offline RL. Understand the algorithmic building blocks.

Start learning

Part II: World Models¶

Learn how agents build internal representations of the world — from video prediction to learned dynamics models to modern foundation world models.

Explore

Part III: Embodied AI¶

Locomotion control, loco-manipulation, teleoperation systems, and data collection strategies for real-world robot learning.

Dive in

Part IV: Distributed RL¶

Scale RL training — from A3C to IMPALA to SEED RL. Understand the systems and frameworks that enable large-scale RL experiments.

How to Use This Resource¶

This resource is designed to be read both linearly (as a textbook) and selectively (as a reference). We recommend:

If you're new to RL: Start with Part I and read sequentially through Key Concepts, Algorithm Taxonomy, and then the individual algorithm pages.
If you're starting embodied AI research: Read Part I for RL foundations, then jump to Part III for embodied-specific content.
If you're building RL systems: Focus on Part IV for distributed architectures and scaling guides.
If you want research guidance: Check out Becoming a Researcher for advice on developing research taste and executing projects.

Each section includes:

Conceptual explanations with mathematical rigor
Key paper references with brief descriptions
Connections between topics across sections
Exercises to test and deepen understanding

Contributing¶

This is a living document. Contributions, corrections, and suggestions are welcome — see the GitHub repository for details.