Fundamentals of Machine Learning

Python · NumPy

A four-exercise homework exam for the Fundamentals of Machine Learning course at the University of Camerino, each task fitting a model to a provided dataset and reconstructing the underlying function or class over a regular grid. Every model is implemented from scratch in plain NumPy — no scikit-learn or other ML library — following the basis-function, probabilistic generative, and logistic regression techniques covered in the course.

The repo contains one self-contained script per exercise, the input datasets supplied for the exam, the resulting predictions, and a full written report with data analysis, modelling choices, cross-validation results, and figures.

How it works

• Exercise 1 — 1D regression — fits a hidden discontinuity in regression1.csv as a polynomial plus a single step basis function (the limiting case of a sigmoidal basis), with polynomial degree chosen by repeated k-fold cross-validation.
• Exercise 2 — 3-class classification — a probabilistic generative model on classification1.csv: class-conditional Gaussians with a shared covariance matrix, selected over per-class covariances by leave-one-out cross-validation, combined with Bayes' theorem for linear decision boundaries.
• Exercise 3 — 2D regression — fits regression2.csv with a bivariate polynomial basis (z = Σ w_ij x^i y^j) by least squares, total degree chosen by cross-validation.
• Exercise 4 — non-linear classification — fits classification2.csv, a non-linearly-separable 2-class problem, via logistic regression on a polynomial feature expansion, solved with Newton-Raphson (IRLS) and a small L2 term for numerical stability.

Contents

• report/ — full write-up covering data analysis, modelling choices, cross-validation results and figures for all four exercises.
• scripts/exercise{1,2,3,4}.py — self-contained scripts, one per exercise, each reading its input dataset and writing the corresponding resultN.csv.
• data/result{1,2,3,4}.csv — model predictions generated by the scripts.

Requirements & installation

Requires only NumPy (`pip install numpy`). Each script is run from a folder containing its input dataset, e.g. `python3 exercise1.py` reads regression1.csv and writes result1.csv. All cross-validation splits use a fixed random seed, so re-running any script reproduces the same selected model complexity and the same resultN.csv, up to floating-point rounding.