Note: This is a Rust project based on by Andrej Karpathy's micrograd. It's a hobby project for practicing backpropagation and Rust. This is purely educational and not intended for real use—it operates on scalar values only, making it extremely inefficient compared to production autograd libraries that use vectorized operations.

---

rustgrad

A tiny scalar-valued autograd engine with a PyTorch-like neural network library on top. Implements backpropagation (reverse-mode autodiff) over a dynamically built DAG. The entire autograd engine is only about 300 lines of code.

The DAG operates over scalar values only, meaning we decompose each neuron into individual tiny additions and multiplications. While inefficient, this is sufficient to build entire deep neural networks for binary classification and demonstrates the core mechanics of backpropagation.

Example usage

Scalar operations and backprop:

The core Value type wraps scalars and tracks computational history for automatic differentiation. Call backward() to compute gradients:

use rustgrad::Value;

let a = Value::new(2.0);
let b = Value::new(3.0);
let c = &a * &b;
c.backward();
assert_eq!(c.item(), 6.0);
assert_eq!(a.grad(), 3.0);  // dc/da = b
assert_eq!(b.grad(), 2.0);  // dc/db = a

Activation functions:

Common activation functions like tanh, relu, and sigmoid are supported:

let x = Value::new(0.5);
let y = x.tanh();
y.backward();
println!("{}", y.item());     // 0.462
println!("{}", x.grad());     // 0.786, i.e. d/dx tanh(x) = 1 - tanh²(x)

Building a simple neural network:

The MLP (Multi-Layer Perceptron) class lets you quickly define feedforward networks:

use rustgrad::MLP;

let model = MLP::new(3, &[4, 4, 1]);  // 3 inputs, two hidden layers of 4, 1 output
let x = vec![2.0, 3.0, -1.0];
let y = model.forward_floats(&x);
println!("{}", y[0].item());  // forward pass

Training a neural net

The examples/classify_2d.rs provides a complete example of training a 2-layer MLP for binary classification. The demo:

• Initializes a neural network using rustgrad::MLP with rustgrad::Linear layers
• Uses a simple SVM "max-margin" binary classification loss
• Trains on the moon dataset with two 16-node hidden layers
• Visualizes the learned decision boundary

The resulting decision boundary shows the network successfully learning to separate the two classes:

Running the demo

To see the classification example in action, which will train the network and generate a visualization:

cargo run --example classify_2d

Running tests

To verify the autograd engine and neural network implementation work correctly:

cargo test

Credits

This project is based on Andrej Karpathy's excellent micrograd, a minimal autograd engine and neural network library. The architecture and API design closely follow the original Python implementation.

License

MIT