Muhammad Imaz Anwar

Engineering lessons
Learn by building

Free learning playlist

Artificial Intelligence: Math to Production

A self-contained AI engineering course from mathematical intuition and Python implementations to transformers, LLM systems, evaluation, and production.

Audience
Software developers with basic Python; no advanced mathematics required.
Publishing
New AI lesson every Tuesday
Progress
4 published · 37 planned
Start with lesson 1 →

COMPLETE CURRICULUM / REFERENCES

Everything this course will cover

The complete curriculum is visible below—including unpublished lessons. Status labels distinguish what is available, what comes next, and what remains planned.

PublishedNext lessonPlanned

Books and primary references

  1. The complete 22-topic AI mathematics syllabus defines the sequence and its engineer-first teaching progression.
  2. Mathematics for Machine Learning supplies the linear algebra, geometry, calculus, probability, optimization, and dimensionality-reduction foundation.
  3. Deep Learning by Goodfellow, Bengio, and Courville supplies neural-network theory, regularization, optimization, and representation-learning depth.
  4. Build a Large Language Model (From Scratch) by Sebastian Raschka supplies the implementation path from text preparation and attention through GPT construction, pretraining, classification fine-tuning, and instruction fine-tuning.
  5. AI Engineering by Chip Huyen supplies the production path: foundation-model evaluation, prompting, RAG, agents, fine-tuning, inference optimization, architecture, feedback, monitoring, and product trade-offs.
  6. Official documentation and original papers verify current APIs, algorithms, numerical behavior, and production claims.

Implementation layer

Every mathematics and model-building lesson includes a Python lab. Labs begin with NumPy or pure Python, then show the PyTorch or production-library equivalent. Published lab files live under /labs/ and are structured for later extraction into a standalone course repository.

Module 01

Numerical foundations and linear algebra

7 lessons

Represent data as numbers, reason about shapes, and understand the geometry behind modern AI.

What this module covers

  • numerical Python, NumPy arrays, PyTorch tensors, dtype and device
  • scalars, vectors, matrices, tensors, bases, span, rank, and linear independence
  • matrix multiplication, broadcasting, axes, linear systems, least squares, and conditioning
  • norms, distances, similarity, transformations, eigenvalues, SVD, low-rank approximation
  • PCA, t-SNE, and UMAP for compression, visualization, and representation debugging

Reference spine: AI mathematics syllabus · Mathematics for Machine Learning · NumPy and PyTorch documentation

  1. 01Linear Algebra Intuition for AI EngineersPublished · 10 minRead lesson →
  2. 02Vectors, Matrices, and OperationsPublished · 10 minRead lesson →
  3. 03Linear Systems and Least SquaresPublished · 13 minRead lesson →
  4. 04Norms, Distances, and SimilarityPublished · 11 minRead lesson →
  5. 05Matrix Transformations and EigenvaluesNext lessonUpcoming
  6. 06SVD and Low-Rank StructurePlannedUpcoming
  7. 07PCA, t-SNE, and UMAPPlannedUpcoming

Module 02

Calculus and optimization

4 lessons

Understand how models measure error, propagate credit, and update millions of parameters.

What this module covers

  • limits, derivatives, partial derivatives, Jacobians, gradients, and directional change
  • chain rule, computation graphs, manual backpropagation, vector-Jacobian products, and autograd
  • gradient descent, SGD, momentum, RMSProp, Adam, AdamW, schedules, clipping, and weight decay
  • convex sets and functions, curvature, Hessian intuition, constraints, and why deep-learning optimization is non-convex

Reference spine: AI mathematics syllabus · Mathematics for Machine Learning · Deep Learning · PyTorch autograd documentation

  1. 08Derivatives and Gradients for Machine LearningPlannedUpcoming
  2. 09Chain Rule, Backpropagation, and AutodiffPlannedUpcoming
  3. 10The Gradient Descent FamilyPlannedUpcoming
  4. 11Convex Optimization and Loss LandscapesPlannedUpcoming

Module 03

Probability, statistics, and information

6 lessons

Reason about uncertainty, evidence, losses, sampling, and model evaluation.

What this module covers

  • random variables, joint/marginal/conditional probability, expectation, variance, covariance, and common distributions
  • descriptive and inferential statistics, confidence intervals, hypothesis tests, bias, variance, leakage, and experiment reliability
  • Bayesian updating, priors, likelihoods, posteriors, calibration, and decision thresholds
  • entropy, cross-entropy, KL divergence, mutual information, language-model loss, and distillation
  • categorical sampling, temperature, top-k, top-p, rejection, importance, Monte Carlo, MCMC basics, and stochastic processes

Reference spine: AI mathematics syllabus · Mathematics for Machine Learning · Deep Learning · original method papers

  1. 12Probability and DistributionsPlannedUpcoming
  2. 13Statistics for Machine LearningPlannedUpcoming
  3. 14Bayes' Theorem and Statistical ThinkingPlannedUpcoming
  4. 15Entropy, Cross-Entropy, and KL DivergencePlannedUpcoming
  5. 16Sampling MethodsPlannedUpcoming
  6. 17Stochastic ProcessesPlannedUpcoming

Module 04

Advanced mathematical tools

5 lessons

Connect tensor computation, numerical behavior, signals, graphs, and complex-valued representations to AI systems.

What this module covers

  • reshape, view, transpose, permute, einsum, gather, scatter, masking, batching, and attention tensor layouts
  • floating-point representation, overflow, underflow, cancellation, stable softmax/log-sum-exp, mixed precision, FP16, BF16, and loss scaling
  • complex arithmetic, Euler's formula, phasors, and complex-valued representations
  • signals, frequency space, DFT, FFT, convolution theorem, spectral features, audio and image applications
  • graphs, adjacency and Laplacian matrices, message passing, spectral methods, GNNs, and graph-based retrieval

Reference spine: AI mathematics syllabus · Deep Learning · NumPy/PyTorch documentation · original numerical and graph-learning papers

  1. 18Tensor OperationsPlannedUpcoming
  2. 19Numerical Stability and PrecisionPlannedUpcoming
  3. 20Complex Numbers for AIPlannedUpcoming
  4. 21The Fourier TransformPlannedUpcoming
  5. 22Graph Theory for Machine LearningPlannedUpcoming

Module 05

Machine learning and deep learning

4 lessons

Build learning algorithms from data preparation and classical models through neural networks and training diagnostics.

What this module covers

  • problem framing, datasets, labels, features, splits, preprocessing, baselines, and metrics
  • linear/logistic regression, losses, regularization, classification, trees, ensembles, clustering, and evaluation
  • neurons, MLPs, activations, initialization, normalization, dropout, forward/backward passes, and optimizers
  • CNN and sequence-model foundations, training loops, checkpoints, overfitting, gradient diagnostics, and reproducibility

Reference spine: Deep Learning · Mathematics for Machine Learning · scikit-learn and PyTorch documentation

  1. 23Framing a Machine Learning ProblemPlannedUpcoming
  2. 24Regression and Classification from ScratchPlannedUpcoming
  3. 25Neural Networks from ScratchPlannedUpcoming
  4. 26Training and Debugging Neural NetworksPlannedUpcoming

Module 06

Transformers and language models

5 lessons

Implement the path from tokens and embeddings to attention, transformer blocks, training, and decoding.

What this module covers

  • text collection, tokenization, vocabulary, data loaders, token and positional embeddings
  • causal self-attention, masks, stable softmax, multi-head attention, tensor shapes, and efficient kernels
  • Transformer blocks, residual streams, normalization, feed-forward networks, logits, and parameter accounting
  • pretraining objectives, training loops, checkpoints, evaluation, loss curves, and weight loading
  • decoding, temperature/top-k/top-p, KV caches, context limits, classification fine-tuning, and instruction fine-tuning

Reference spine: Build a Large Language Model (From Scratch) · Attention Is All You Need · GPT-family papers · PyTorch documentation

  1. 27Tokenization and EmbeddingsPlannedUpcoming
  2. 28Attention from First PrinciplesPlannedUpcoming
  3. 29Build a Transformer BlockPlannedUpcoming
  4. 30Train a Small Language ModelPlannedUpcoming
  5. 31Decoding, Sampling, and KV CachesPlannedUpcoming

Module 07

AI engineering and production

6 lessons

Turn model capabilities into evaluated, observable, secure, and cost-aware products.

What this module covers

  • foundation-model capabilities, model selection, APIs, structured outputs, prompt and context design
  • embedding systems, chunking, retrieval, reranking, RAG evaluation, citations, and vector stores
  • tool calling, workflow state, agents, memory, planning, permissions, guardrails, and human approval
  • evaluation design, task-specific metrics, judges, test sets, error analysis, online feedback, safety, and red teaming
  • fine-tuning data, LoRA, preference methods, inference optimization, batching, caching, quantization, serving, observability, latency, cost, security, and drift

Reference spine: AI Engineering · Build a Large Language Model (From Scratch) · official provider/framework documentation · original RAG, LoRA, evaluation, and serving papers

  1. 32Model APIs and Structured OutputsPlannedUpcoming
  2. 33Embeddings, Vector Search, and RAGPlannedUpcoming
  3. 34Tool Calling and AI AgentsPlannedUpcoming
  4. 35Evaluating AI SystemsPlannedUpcoming
  5. 36Fine-Tuning and LoRAPlannedUpcoming
  6. 37Production LLM ServingPlannedUpcoming