Phase 528 DaysAdvanced
Phase 5 — Generative AI
Build production-grade LLM applications using RAG, fine-tuning, and prompt engineering — with evaluation, safety, and cost control built in from the start.
- Build a RAG pipeline that is grounded, measurable, and failure-aware.
- Fine-tune an open LLM with QLoRA on a custom task under low-VRAM constraints.
- Evaluate generative outputs for quality, groundedness, and safety.
⚡ Must Know
- LLM Architecture — decoder-only, autoregressive
- Tokenization — BPE, SentencePiece
- Prompt Engineering — zero-shot, few-shot, CoT
- Embeddings + Semantic Similarity
- Vector Databases — Chroma, FAISS, Pinecone
- RAG Pipeline — chunk, embed, store, retrieve, generate
- LangChain + LlamaIndex basics
- OpenAI API + Function Calling
- Hugging Face Transformers — pipeline API
- Fine-Tuning vs RAG vs Prompting — decision matrix
- LoRA + QLoRA — PEFT methods
- LLM Evaluation — ROUGE, BLEU, LLM-as-judge
✨ Good to Know
- Diffusion Models — DDPM, Stable Diffusion
- GANs + VAEs — generative architectures
- Ollama + vLLM — local LLM hosting
- RLHF — alignment concepts
- LLM Scaling Laws — compute, data, params
- Anthropic Claude API + system prompts
📚 Resources
Hugging Face NLP Course
Best resource for transformers, tokenizers, and fine-tuning.
huggingface.co/learn ↗PEFT Docs — LoRA Guide
Official guide to parameter-efficient fine-tuning methods.
huggingface.co/docs/peft ↗DeepLearning.AI — LLM Short Courses
Short, focused courses on RAG, agents, and fine-tuning.
deeplearning.ai ↗🏗️ Projects
RAG Document Chatbot
Q&A on custom docs with source-grounded responses and failure handling.
Fine-tune Llama 3 (QLoRA)
Adapt an open LLM with parameter-efficient fine-tuning under low VRAM.
Semantic Search Engine
Embedding-based search with vector indexing and retrieval quality metrics.