v0.5.27

Mooncake v0.5.27

Diff since v0.5.26

Merged pull requests:

• Fix tangent_type for Union{NoRData, RData{...}} (#1133) (@yebai)
• drop Julia 1.11 CI support and refresh 1.12 doctests (#1138) (@yebai)
• Pointer Handling for zero_tangent (#1139) (@AstitvaAggarwal)
• foreigncall zero derivatives for Forward over reverse (#1143) (@AstitvaAggarwal)
• Fix ci failures (#1144) (@AstitvaAggarwal)
• Updates for CUDA + cuDNN 6 (#1148) (@kshyatt)
• Add cached value_and_jacobian!! interface. (#1153) (@yebai)
• Disable Turing benchmarks until DI is removed from Turing.jl hard deps (#1155) (@yebai)
• cleanup for codecov (#1158) (@AstitvaAggarwal)

Closed issues:

• Add support for reverse mode AD of multi-threaded map! operation (#791)
• Safe usage of zero_tangent for Ptr{T} (#999)
• missing tangent_type method (#1130)
• How to do inplace gradient in Lux.jl (#1137)
• Add value_and_jacobian (#1145)
• Unable to differentiate a simple function involving arbitrary precision numbers (#1146)
• CUDA failures on Julia 1.12 (#1154)
• Avoid COV_EXCL_START and COV_EXCL_STOP markup (#1157)

v0.5.26

Mooncake v0.5.26

Diff since v0.5.25

Merged pull requests:

• Some more Lux tests, minor changes (#1017) (@AstitvaAggarwal)
• Drafting some developer facing agent skills (#1027) (@sunxd3)
• Add forward threading rules and explicit reverse-mode error (#1125) (@yebai)
• Respect primitive-rule dispatch in inspect_ir (#1126) (@yebai)
• Fix friendly tangents for NamedTuple args (#1129) (@sunxd3)
• Update AGENTS.md guidance and refactor agent skills (#1134) (@yebai)
• non differentiable support for pkg, ext loading (#1135) (@AstitvaAggarwal)
• nfwd: canonical slot traversal via _fold_slots / _unfold_slots (#1136) (@yebai)

Closed issues:

• Benchmarking Utility Functionality (#364)
• Forwards-Mode Rules for nnlib and friends (#542)
• Threads support (#570)
• Mooncake does not like sparse cholesky (#698)
• coding agent skills for autograd IR transform (#1004)
• friendly_tangents crashes on NamedTuple arguments in tangent_to_friendly!! (#1128)

v0.5.25

Mooncake v0.5.25

Diff since v0.5.24

This release has been identified as a backport.
Automated changelogs for backports tend to be wildly incorrect.
Therefore, the list of issues and pull requests is hidden.

v0.5.24

Mooncake v0.5.24

Diff since v0.5.23

Merged pull requests:

• Add AD rules for det/logdet/logabsdet on Symmetric matrices (fixes #819) (#1055) (@yebai)
• fix getrs! Core.Box instability + perf bounds (#1058) (@yebai)
• batched_matmul_fallback handling for NNlib.Batched Transpose, Adjoint (#1067) (@AstitvaAggarwal)
• Fix primal_ir to work with non-differentiable code (#1068) (@yebai)
• Show source location of failing method in MooncakeRuleCompilationError (#1069) (@yebai)
• Fix DI second-order test to only run intended variant per CI job (#1070) (@yebai)
• fix(ndual): branchless min/max/clamp/flipsign/copysign; cache tanh; optimize Complex sqrt (#1071) (@yebai)
• fix(cuda): free NDual arrays in sum(f,x) rrules; factor pullback helper; fix allocs (#1072) (@yebai)
• Extract repeated BLAS guard blocks into helpers; pre-compute cumprod inv (#1073) (@yebai)
• Fix three minor issues in GPU mutation rules (PR 1064 follow-up) (#1074) (@yebai)
• Use _model_name in Lux testset titles (PR 1065 follow-up) (#1075) (@yebai)
• Add batch-size-1 broadcast test coverage for batched_matmul_fallback (#1076) (@yebai)
• fix(@zero_derivative): support parametric Vararg{T} and Vararg{T,N} (#1078) (@yebai)
• feat: add Mooncake.stop_gradient to block gradient flow (#1079) (@yebai)
• LuxLib fix followup (#1080) (@AstitvaAggarwal)
• fix(1.10): guard DebugRRule invalid CoDual specializations (#1085) (@yebai)
• Move world-age DI test and remove second-order ext suite (#1088) (@yebai)
• refactor: simplify count_allocs using @generated (#1093) (@yebai)
• refactor: replace Tcount_allocs barrier with inlinable __count_allocs (#1094) (@yebai)

Closed issues:

• GPU Rules and tests (#278)
• Integration Benchmarks (#520)
• Handling recursive types automatically (#602)
• Source location in rrule!! stack traces (#649)
• Mooncake.primal_ir errors if primal code contains non-differentiable code (#668)
• Implement all benchmarking cases for gradben.ch (#795)
• Missing rrule for logdet of symmetric matrices (#819)
• more informative error message for Vararg in @zero_derivative. (#923)
• customised tangent types and testing for ROCArray (#924)
• Inconsistent gradients for forward and reverse mode (#992)
• Reduce use of community CI resources (#1086)

v0.5.23

Mooncake v0.5.23

Diff since v0.5.22

Merged pull requests:

• PR 2a: GPU AD infrastructure — NDual type for GPU chunk-mode forward-mode AD (#1060) (@yebai)
• PR 2b: GPU AD rules — broadcast forward pass and sum/sum(f, x) rules (#1061) (@yebai)
• PR 3: GPU AD rules — BLAS, reductions, indexing, and unsupported-op errors (#1063) (@yebai)
• PR 4: GPU AD rules — mutation (fill!, unsafe_copyto!) (#1064) (@yebai)
• PR 5: GPU integration tests — Flux, Lux, and Buildkite pipeline (#1065) (@yebai)
• SLEEFPirates, LuxLib silent errors (#1066) (@AstitvaAggarwal)

v0.5.22

Mooncake v0.5.22

Diff since v0.5.21

Merged pull requests:

• PR 1: register non-differentiable CUDA types and fix Task field handling (#1057) (@yebai)
• Mark some CUSOLVER types inactive (#1062) (@kshyatt)

v0.5.21

Mooncake v0.5.21

Diff since v0.5.20

Merged pull requests:

• Add BFloat16 support via Core.BFloat16 tangent infrastructure and BFloat16s extension (#1042) (@yebai)
• Fix DebugFRule → DebugRRule in FoRCache and document aliasing (#1051) (@yebai)
• Fix syrk!/herk! rules to support vector argument (closes #786) (#1053) (@yebai)
• Add Flux integration tests for TEST_MODELS (#1054) (@yebai)

Closed issues:

• issue with mul!(A, x, x') (#786)
• missing support for BFloat16 (#935)

v0.5.20

Mooncake v0.5.20

Diff since v0.5.19

Merged pull requests:

• NNlib activation function rules (#1043) (@AstitvaAggarwal)
• Fix differentiating GPU Dense layer (#910) (#1044) (@yebai)
• Dual Tangent testing + Abstract Vararg / NTuple UnionAll handling. (#1047) (@AstitvaAggarwal)
• refactor: use arrayify for Adjoint/Transpose accumulation in NNlib rrules (#1048) (@yebai)
• Widen SupportedArray to include Adjoint/Transpose (#1049) (@yebai)
• bench: run incremental GC between benchmark samples (#1050) (@yebai)
• Bump version from 0.5.19 to 0.5.20 (#1052) (@AstitvaAggarwal)

Closed issues:

• missing GPU rules required by Flux (#910)
• Failure with basic Flux Chain on GPU (#996)

v0.5.19

Mooncake v0.5.19

Diff since v0.5.18

Merged pull requests:

• Minor fixes for CuArray tangent rules (#1045) (@yebai)
• zero adjoints for Lux (#1046) (@AstitvaAggarwal)

v0.5.18

Mooncake v0.5.18

Diff since v0.5.17

Merged pull requests:

• Merge blas + blas_level_3; split all BLAS tests by precision (#1037) (@yebai)
• 1035 logsumexp subarray (#1038) (@GBarnsley)
• Add tangent_type for CUBLAS.cublasComputeType_t (#1041) (@yebai)

Closed issues:

• Issues differentiating LogExpFunctions.logsumexp (#1035)
• nested AD for more backends. (#1039)