Laplace Approximation

[hmd101]

Summary

Implement Laplace approximation: Gaussian approximation to posterior centered at MAP with covariance from Hessian inverse.

Mathematical Background

Algorithm:

1. Find MAP: θ_MAP = argmax p(θ|data)
2. Compute Hessian: H = -∇²log p(θ|data)|_{θ=θ_MAP}
3. Approximate: p(θ|data) ≈ N(θ_MAP, H^{-1})

Assumptions:

• Posterior is unimodal
• Approximately Gaussian near mode
• Symmetric (no heavy skew)

Assumptions:
Fails when multimodal, but might be useful for placing more probes around a given reference point where threshold ellipses currently are misaligned.

Implementation Tasks

Task 1: Core Laplace Implementation

File: src/psyphy/inference/laplace.py

Requirements:

• Create LaplaceApproximation class implementing InferenceEngine
• Reuse MAPInference to find mode
• Compute Hessian via jax.hessian()
• Invert Hessian with regularization for numerical stability
• Return LaplacePosterior object

API:

from psyphy.inference import LaplaceApproximation

laplace = LaplaceApproximation(
    map_kwargs={'num_steps': 1000, 'learning_rate': 0.01},
    regularization=1e-4  # Added to diagonal before inversion
)
posterior = laplace.fit(model, data)