Open-source PyTorch curvature library rewritten from scratch after 8 years

pytorch-hessian-eigenthings has been cleaned up and reorganized into a v1.0.0a1 alpha. It computes eigenvalues and eigenvectors for the Hessian, Generalized Gauss-Newton (GGN), and empirical Fisher of PyTorch models, analyzing curvature structure without materializing the full Hessian.

The core algorithms are Lanczos, stochastic power iteration, Hutch++-based trace estimation, and Stochastic Lanczos Quadrature. It operates using only Hessian-vector products, reducing memory bottlenecks even on large models, and supports HuggingFace and TransformerLens transformers as well.

For LM-scale tasks, hf_lm_loss_of_output() automatically selects a Triton fused kernel on CUDA environments, or a torch.compile path otherwise. According to the docs, this path improves speed and peak memory compared to eager.

In the new API, the 0.x interface has been removed, and existing users should pin to hessian-eigenthings==0.0.2. param_filter allows analyzing only specific parameter blocks, and a finite-difference HVP path is also provided for environments like FSDP where double-backward is costly.