project

Predictive Modeling & Data Insights Analysis

An empirical comparison of five supervised learning algorithms — Logistic Regression, KNN, SVM (RBF kernel), Random Forest, and Gradient Boosted Trees — across four UCI datasets, evaluating how well each model generalizes under different training set sizes.

Pythonscikit-learnpandasNumPymatplotlib

approach

Built a shared preprocessing + model pipeline (median/mode imputation, standardization, one-hot encoding) for all five classifiers.
Tuned hyperparameters per classifier with GridSearchCV and 5-fold stratified cross-validation.
Evaluated each dataset/classifier pair across three train/test splits (20/80, 50/50, 80/20) and three random seeds — 180 experiments in total.
Compared training, cross-validated, and test accuracy to analyze generalization and overfitting.

key findings

Gradient Boosted Trees and Random Forest most consistently achieved the top test accuracy, with SVM (RBF) close behind.
Logistic Regression was a surprisingly strong, stable baseline across all four datasets.
KNN tended to overfit — high training accuracy but the weakest test accuracy, especially on larger datasets.
More training data almost always helped, with the largest gains on the harder tasks and diminishing returns once a dataset was already large.

results

Bar chart of test accuracy by classifier on the Breast Cancer dataset at 80% training fraction — Breast Cancer dataset — SVM (RBF) and Logistic Regression lead at ~97% test accuracy.

Bar chart of test accuracy by classifier on the Adult dataset at 80% training fraction — Adult dataset — Gradient Boosted Trees and Random Forest lead, KNN trails.

Line chart of cross-validated accuracy vs regularization strength C for logistic regression on the Adult dataset — Hyperparameter tuning — validation accuracy peaks at moderate regularization strength (C).

view code on GitHub →

Full methodology, tables, and discussion are in the written report on GitHub.