This project implements logistic regression for binary classification. It covers both standard logistic regression and regularized logistic regression with polynomial features for non-linear decision boundaries.
Logistic regression uses the sigmoid function to model the probability of class membership:
h(x) = 1 / (1 + exp(-theta' * x))
The cost function uses cross-entropy loss:
J(theta) = -(1/m) * sum(y*log(h(x)) + (1-y)*log(1-h(x)))
Regularization adds a penalty term (lambda/2m) * sum(theta_j^2) to prevent overfitting when using polynomial features.
| File | Description |
|---|---|
sample2.m |
Main script: logistic regression for student admission prediction |
sample2_reg.m |
Main script: regularized logistic regression for microchip QA |
sample2_reg_var_lambda.m |
Varying lambda to observe over/underfitting |
costFunction.m |
Logistic regression cost and gradient |
costFunctionReg.m |
Regularized cost and gradient |
sigmoid.m |
Sigmoid activation function |
plotData.m |
Plots 2D classification data |
plotDecisionBoundary.m |
Plots the learned decision boundary |
mapFeature.m |
Maps features to polynomial features |
predict1.m |
Predicts class using learned parameters |
ex2data1.txt |
Dataset: student exam scores and admission |
ex2data2.txt |
Dataset: microchip test results and pass/fail |
animation.gif |
Animation of decision boundary vs. lambda |
- Student Admission: A student with scores 45 and 85 has a ~77.6% admission probability. Training accuracy: 89%.
- Microchip QA: With lambda=1, training accuracy is ~83.1%. The decision boundary clearly shows how regularization controls model complexity.
Left: Linear decision boundary for student admission. Right: Non-linear decision boundary for microchip classification using polynomial features.
Exercises from Andrew Ng's Machine Learning course on Coursera, completed by Keivan Hassani Monfared.