CSE273 · Foundations of Machine Learning

The foundations of machine learning, on one real dataset.

Six notebooks, one per unit of CSE273. Every idea is computed on the Kaggle Playground S6E4 Predicting Irrigation Need data (630,000 fields), and every formula is checked against a library or a numerical answer: the bias–variance split is measured, gradients are verified with finite differences, gradient descent, Momentum, RMSProp and Adam are written from scratch, and PAC bounds are tested empirically.

The reusable parts live in a small tested package, groundwork; the notebooks run unchanged on Kaggle.

6notebooks, one per unit
57syllabus topics mapped
4optimisers written from scratch
16unit tests

Notebooks

01 · UNIT I

Machine learning foundations and NumPy

12 syllabus topics

02 · UNIT II

Pandas for data handling

7 syllabus topics

03 · UNIT III

Exploratory data analysis

8 syllabus topics

04 · UNIT IV

Linear algebra, probability and calculus

9 syllabus topics

05 · UNIT V

Statistics for ML and optimisation

11 syllabus topics

06 · UNIT VI

Statistical learning theory

10 syllabus topics

Results

Bias² and variance measured with 30 bootstrap fits per tree depth.
Correlation of every numeric feature with irrigation need.
Bayes' theorem, one piece of evidence at a time.
Gradient descent, Momentum, RMSProp and Adam (from scratch) on the Rosenbrock valley.
500 confidence intervals: about 95% contain the true mean.
Empirical vs true risk under ERM, as the sample grows.

Run it yourself

git clone https://github.com/LuShadowX/Groundwork && cd Groundwork
pip install -e ".[dev]"
kaggle competitions download -c playground-series-s6e4 -p data && unzip data/*.zip -d data
pytest                      # 16 tests
jupyter lab notebooks/      # or open any notebook on Kaggle with the competition attached

Course map

All 57 topics of the CSE273 syllabus, unit by unit, with the notebook section that covers each one.

Unit I: Machine Learning Foundation12 topics

Syllabus topicWhere
Introduction to machine learning01 § 1
AI vs machine learning vs data science01 § 2
Supervised, unsupervised and reinforcement learning01 § 3
Parametric vs non-parametric models01 § 4
Generative vs discriminative models01 § 4
Machine learning workflow01 § 5
Train–test split, overview of evaluation metrics01 § 6
Overfitting and underfitting01 § 6
Bias–variance trade-off (measured with 30 bootstrap fits)01 § 6
Applications and use cases01 § 7
NumPy: why NumPy, creating arrays, operations01 § 8
Broadcasting rule, matrix operations, random module01 § 8

Unit II: Pandas for Data Handling7 topics

Syllabus topicWhere
Pandas Series and DataFrame02 § 1
Reading and writing CSV files02 § 2
Inspecting datasets02 § 3
Selection: loc vs iloc; boolean filtering02 § 4
Line, bar, histogram, scatter and pie charts02 § 5
Customising plots02 § 6
Subplots02 § 7

Unit III: Exploratory Data Analysis8 topics

Syllabus topicWhere
Types of data (quantitative, qualitative)03 § 1
Loading data and initial inspection03 § 2
Univariate analysis: histogram, box plot, count plot03 § 3
Bivariate analysis: scatter, line, violin plots03 § 4
Correlation heatmaps and multicollinearity (VIF)03 § 5
Skewness and kurtosis03 § 6
Patterns, anomalies (outliers) and trends03 § 7
EDA workflow integration03 § 8

Unit IV: Linear Algebra and Calculus9 topics

Syllabus topicWhere
Data structures: scalar, vector, matrix, tensor04 § 1.1
Eigenvalues and eigenvectors (power iteration from scratch)04 § 1.2
Random variables and probability distributions04 § 2.1–2.2
Statistical measures04 § 2.3
Types of probability: joint, marginal, conditional04 § 2.4
Bayes' theorem and inference04 § 3.1
Likelihood vs probability04 § 3.2
Functions, partial derivatives and the gradient04 § 4.1–4.3
The chain rule04 § 4.4

Unit V: Statistics for ML11 topics

Syllabus topicWhere
Sampling and population05 § 1.1
Hypothesis testing05 § 1.2
Confidence intervals05 § 1.3
Correlation05 § 1.4
Maximum likelihood estimation05 § 2.1
Loss (cost) functions05 § 2.2
Convex vs non-convex functions05 § 3.1
Minima and saddle points05 § 3.2
Gradient descent and the learning rate05 § 4.1–4.2
Variants of gradient descent (batch, stochastic, mini-batch)05 § 4.3
Momentum, RMSProp and Adam (from scratch)05 § 5

Unit VI: Statistical Learning Theory10 topics

Syllabus topicWhere
Well-posed learning problems (T-P-E)06 § 1
Components of a learning system06 § 2
Statistical learning framework06 § 3
Empirical risk minimisation, overfitting and generalisation06 § 4
Inductive bias06 § 5
Role of prior knowledge06 § 6
PAC learning06 § 7
Sample complexity: finite hypothesis space and VC dimension06 § 8
No Free Lunch theorem06 § 9
Choosing algorithms; Occam's razor06 § 10