100% Free · No Signup Required

AI Engineering
Roadmap

16 weeks. ~68 hours. A curated, minimal path through the best free resources on the internet.

Progress 0 / 68h completed

Foundations

W1-4

Training & Optimization

Multimodal & Capstone

W13-16

Foundations

Neural networks, transformers, build GPT from scratch

Weeks 1-4

Core Path

▶

Watch ~1h

3Blue1Brown "Neural Networks" - videos 1-3 only

Visual intuition for parameters, backprop, loss, activation functions

▶

Watch 1h

Karpathy "Intro to Large Language Models"

Full LLM landscape: pretraining, supervised vs self-supervised, scaling laws, RLHF

🛠

Build 2.5h

Karpathy "Building Micrograd" - code along

Build a neural network from scratch. This IS your MNIST-equivalent assignment.

+ Chinchilla Scaling Laws paper + Hugging Face Deep RL course (chapter 1) + Lilian Weng: Contrastive Learning + Karpathy micrograd repo

neural networkslearningscaling lawsLLMs

Core Path

🛠

Build 2.25h

Karpathy "Let's build the GPT Tokenizer" - code along

BPE tokenization from scratch. Code along.

▶

Watch 26min

3Blue1Brown Ch.5 "But what is a GPT?"

Vectorization/embeddings and positional encodings explained visually

▶

Watch 26min

3Blue1Brown Ch.6 "Attention in Transformers"

The attention mechanism - the only attention intro you need

+ Jay Alammar: Illustrated Word2Vec + Hugging Face NLP Course Ch.6 (Tokenizers) + Karpathy minbpe repo

tokensvectorsattentionLLMs

Core Path

📖

Read ~1h

Jay Alammar "The Illustrated Transformer"

Covers Q/K/V, multi-head attention, FFNNs, logits - the entire week in one post

📖

Read ~1h

Sebastian Raschka: Understanding & Coding All Attention Types

All 4 attention types coded side by side

📖

Read ref

Harvard "Annotated Transformer" - skim as reference

Don't read cover to cover. Use as code reference when confused.

+ Umar Jamil: Full transformer video + Peter Bloem: Transformers from Scratch + Lilian Weng: Transformer Family v2.0

transformerLLM internalsAttention

Core Path

🛠

Build 2h

Karpathy "Let's build GPT" - code along

This single video IS the entire week. Build a working transformer from scratch.

+ Umar Jamil: Full encoder-decoder transformer + Karpathy nanoGPT repo + rasbt/LLMs-from-scratch (chapters 3-4) + Jay Alammar: Illustrated GPT-2

transformer codecausal attentionautoregressive

⚡ You can now build a GPT from scratch

Training & Optimization

LLM lifecycle, quantization, fine-tuning with LoRA

Weeks 5-6

Core Path

▶

Watch 2h

Karpathy "Deep Dive into LLMs like ChatGPT"

Full lifecycle: pre-training data, training, post-training (SFT, RLHF), evaluation.

📖

Read 30min

Hugging Face "Illustrating RLHF"

Best visual explanation of the RLHF pipeline

+ Karpathy: Let's reproduce GPT-2 (124M) + Stanford CS324: Large Language Models + Open LLM Leaderboard

llmsystem designdata processingevaluation

Core Path

📖

Read 45min

Maxime Labonne "Introduction to Weight Quantization"

FP16, INT8, INT4, GPTQ, GGUF, AWQ, KV cache - everything in one article

📖

Read 30min

Sebastian Raschka "Practical Tips for Finetuning LLMs Using LoRA"

LoRA/QLoRA explained with practical advice

🛠

Build ~2h

Hugging Face PEFT Quickstart - actually run it

Fine-tune a model using LoRA. This is your coding assignment.

+ DeepLearning.AI: Quantization Fundamentals (free) + DeepLearning.AI: Finetuning LLMs (free) + Axolotl fine-tuning framework

LLM optimizationsquantizationLORAfine-tuning

⚡ You can now fine-tune any open-source LLM

RAG & Retrieval

Vector databases, RAG pipelines, guardrails, safety

Weeks 7-8

Core Path

▶

Watch ~2h

LangChain "RAG From Scratch" - videos 1-5 only

Overview, indexing, retrieval, generation, query translation.

📖

Read 20min

Pinecone HNSW explainer

Vector search / ANN algorithms explained in one article

🛠

Build ~1h

ChromaDB Getting Started - hands-on

Get hands-on with a vector database

+ RAG From Scratch videos 6-14 (advanced) + DeepLearning.AI: Vector Databases (free) + Pinecone Learning Center

RAGVector DBVector Search

Core Path

▶

Watch ~2h

LangChain "RAG From Scratch" - videos 6-10

Reranking, HyDE, query rewriting, routing

📖

Read 30min

Simon Willison: Prompt Injection - first 2 posts

Safety and guardrails essentials

🛠

Build ~3h

DeepLearning.AI "Chat with Your Data" - full course

Build a RAG chatbot. Complete the whole short course.

+ Gandalf prompt injection game + NVIDIA NeMo Guardrails + OWASP Top 10 for LLM Apps

RAGsafetyguardrailscoding

⚡ You can now build production RAG applications

Agents & Production

AI agents, MCP, evals, reasoning models

Weeks 9-12

Core Path

📖

Read ~1h

Anthropic "Building Effective Agents"

LLM vs agent, chaining, orchestration, routing, ReAct - the whole week in one article

📖

Read 45min

Lilian Weng "LLM Powered Autonomous Agents" (sections 1-3)

Planning, memory, and tool use architecture

🛠

Build ~2h

LangGraph Customer Support Agent tutorial

Build a customer support agent.

+ DeepLearning.AI: AI Agents in LangGraph (free) + ReAct paper + Anthropic Cookbook

agentsreActorchestrationcoding

Core Path

📖

Read ~1h

MCP Docs - Quickstart + Core Architecture only

Don't read the whole spec. Quickstart + architecture is enough.

📖

Read 20min

Lilian Weng agent post - Memory section

Short-term, long-term, and episodic memory in agents

🛠

Build ~2h

Build an MCP server - official tutorial

Build a working MCP server using the Python SDK

📖

Read 15min

Google A2A repo README

Skim the Agent-to-Agent protocol overview

+ Mem0 (memory layer for agents) + LangGraph multi-agent tutorials + Microsoft AutoGen

agentic memorymcpcontext engineering

⚡ You can now build and deploy AI agents with memory

Core Path

📖

Read 30min

Hamel Husain "Your AI Product Needs Evals"

Why evals matter and how to build them

📖

Read 30min

Hamel Husain "Creating an LLM-as-Judge"

Evals + LLM-as-judge completely covered

📖

Read 45min

Eugene Yan "Patterns for Building LLM Systems"

Fine-tuning vs prompting vs RAG tradeoffs

🛠

Build ~1.5h

Set up basic evals using DeepEval

Write 3-4 test cases for an LLM output.

+ DeepLearning.AI: Advanced RAG Evals (free) + Ragas (RAG evaluation) + Full Stack LLM Bootcamp

EvalsLLM as Judgehallucinations

Core Path

📖

Read 20min

OpenAI "Learning to Reason with LLMs"

How reasoning models work internally

📖

Read 20min

DAIR.AI Chain-of-Thought guide

Chain-of-thought prompting with examples

📖

Read 20min

OpenAI Reasoning prompting guide

How to prompt reasoning models differently

📖

Read 30min

DeepSeek-R1 paper - abstract + sections 1-3

RL-trained reasoning. Skim only.

🛠

Build ~1h

Try DeepSeek-R1 on Hugging Face

Compare outputs with/without chain of thought

+ DeepLearning.AI: RLHF (free course) + Original CoT paper (Wei et al.) + Anthropic Extended Thinking docs

reasoningchain of thoughtRLHF

⚡ You can now build, evaluate, and ship production AI systems

Multimodal & Capstone

Vision, diffusion, capstone project, career

Weeks 13-16

Core Path

📖

Read 30min

Lilian Weng "From GAN to WGAN" - sections 1-3 only

Get the generator/discriminator concept and move on.

▶

Watch 30min

Yannic Kilcher on CLIP

CLIP bridges vision and language - the key concept

📖

Read 30min

Hugging Face "Vision Language Models Explained"

Multimodal models (GPT-4V, LLaVA, Gemini)

📖

Read 20min

OpenAI Sora technical report - first 3 pages

Video generation architecture. Skim only.

+ Build a GAN (TensorFlow tutorial) + OpenAI CLIP repo + LLaVA repo

clipmultimodalLLMsimages

Core Path

📖

Read 45min

Lilian Weng "What are Diffusion Models?" - sections 1-4

Focus on forward/reverse process intuition.

📖

Read 30min

Jay Alammar "The Illustrated Stable Diffusion"

Visual explanation of text-to-image

🛠

Build ~2h

Hugging Face Diffusion Course - Unit 1 only

Build a diffusion model from scratch. Unit 1 is enough.

+ Umar Jamil: Stable Diffusion from scratch + DeepLearning.AI: How Diffusion Models Work (free) + Stable Diffusion WebUI

DiffusionImagesVideosFlow

Core Path

🛠

Build

Option A: RAG Application (recommended)

Document Q&A with reranking, guardrails, evaluation.

🛠

Build

Option B: AI Agent

Multi-tool agent with search, compute, and reasoning.

🛠

Build

Option C: Fine-tuned Model

Fine-tune an open model for a specific task.

🛠

Build

Deploy on Hugging Face Spaces (free)

Put your project live with a shareable demo.

+ Made With ML (project structure) + Microsoft Generative AI for Beginners

projectreviewshowcase

Core Path

📖

Read 45min

Chip Huyen "Building LLM Applications for Production"

Best practices, cost, latency, limitations - all in one post

📖

Read 30min

Eugene Yan "Patterns for Building LLM Systems"

Re-read focusing on architecture patterns

📖

Read 20min

Shawn Wang "The Rise of the AI Engineer"

What AI engineering is and where the field is going

▶

Watch 1h

Rewatch: Karpathy "Intro to LLMs"

You'll catch things you missed the first time.

+ Interview prep: ML Systems Design + Simon Willison's blog + OWASP LLM Top 10

best practicesLLMsAI Engineering

⚡ You are now an AI Engineer

The 3 Non-Negotiables

If you skip everything else, don't skip these.

Karpathy's YouTube

Weeks 1-5. The foundation. Watch and code along.

Jay Alammar's Blog

Weeks 2-3, 14. Best visual explanations of transformers.

One Project Per Section

Weeks 6, 8, 9, 10, 11, 15. Reading without building is useless.

AI Engineering Roadmap

Foundations

Overview of LLMs

Core Path

Tokenization, Vectorization, and Attention

Core Path

Transformer Architecture Internals

Core Path

Causal Attention + Coding a Transformer

Core Path

Training & Optimization

LLM Training at Scale

Core Path

Quantization and Fine-Tuning

Core Path

RAG & Retrieval

Retrieval Augmented Generation

Core Path

Hands-on RAG Implementation

Core Path

Agents & Production

AI Agents and Tool Calling

Core Path

MCP, Context Engineering, Multi-Agent Systems

Core Path

Evals, AI Applications in Production

Core Path

Reasoning Models

Core Path

Multimodal & Capstone

Image and Video Models

Core Path

Diffusion Based Models

Core Path

Capstone Project

Core Path

AI Engineering Principles

Core Path

The 3 Non-Negotiables

Karpathy's YouTube

Jay Alammar's Blog

One Project Per Section

AI Engineering
Roadmap