Lux: Explicit Parameterization of Deep Neural Networks in Julia

LuxTestUtils LuxTestUtils-v2.3.1

Diff since LuxTestUtils-v2.3.0

Merged pull requests:

• fix: N-dimensional mask and bias support in ReactantExt MultiHeadAttention (#1682) (@Copilot)
• test: reenable mooncake testing for training api (#1683) (@avik-pal)
• fix: correct double activation in cublasLt_fused_dense! generic fallback (#1685) (@Copilot)
• docs: add missing mooncake_gradient_function docstring to LuxTestUtils API page (#1687) (@Copilot)
• fix: host memory leak in cublaslt implementation (#1689) (@avik-pal)
• chore: bump codecov/codecov-action from 5 to 6 (#1691) (@dependabot[bot])
• chore: bump crate-ci/typos from 1.44.0 to 1.45.0 (#1692) (@dependabot[bot])
• feat: allow new CUDA versions (#1694) (@avik-pal)
• Add continue-on-error + input toggles for telemetry and cache in CommonCI (#1695) (@ChrisRackauckas-Claude)
• fix: loading of CUDAExt for CUDA v6 (#1696) (@avik-pal)
• fix: update to new GPUArrays RNG API (#1697) (@avik-pal)
• fix(WeightInitializers): update GPUArrays RNG handling (#1698) (@avik-pal)
• chore: bump julia-actions/setup-julia from 2 to 3 (#1699) (@dependabot[bot])
• chore: bump crate-ci/typos from 1.45.0 to 1.45.1 (#1700) (@dependabot[bot])
• fix: cudnn.functional was removed (#1701) (@avik-pal)
• fix: cublaslt on cuda v6 (#1702) (@avik-pal)
• test: don't test Mooncake for now (#1703) (@avik-pal)
• MLDataDevices: allow RecursiveArrayTools v4 (#1707) (@ChrisRackauckas-Claude)
• fix: RNG not present in new cuRAND (#1708) (@avik-pal)
• chore: bump crate-ci/typos from 1.45.1 to 1.46.0 (#1710) (@dependabot[bot])
• fix: CUDAExt for WeightInitializers (#1711) (@avik-pal)
• fix: adjust/adjust! broken for ReactantOptimiser(OptimiserChain(...)) (#1713) (@Copilot)
• chore: bump crate-ci/typos from 1.46.0 to 1.46.1 (#1715) (@dependabot[bot])
• Fix docs homepage Tutorials CTA to use canonical /tutorials/ route (#1719) (@Copilot)
• chore: bump crate-ci/typos from 1.46.1 to 1.46.2 (#1720) (@dependabot[bot])
• chore: bump crate-ci/typos from 1.46.2 to 1.46.3 (#1721) (@dependabot[bot])
• chore: bump crate-ci/typos from 1.46.3 to 1.47.0 (#1724) (@dependabot[bot])
• chore: bump crate-ci/typos from 1.47.0 to 1.47.2 (#1726) (@dependabot[bot])
• chore: bump codecov/codecov-action from 6 to 7 (#1727) (@dependabot[bot])
• [LuxTestUtils] Move mooncake to an extension (#1743) (@wsmoses)

Closed issues:

• Irregular RAM usage under large amount of epochs on gpu (#872)
• Memory leak in Dense layer with CUDA (#1230)
• Mooncake is completely busted in recent versions (#1679)
• Attention assumes 2D mask when using Reactant (#1681)
• Possible double activation in LuxLib.Impl.cublasLt_fused_dense! (#1684)
• Documentation build is broken (#1686)
• MLDataDevices prevents updating to SciMLBase@3 (#1705)
• Upcoming modification in Mooncake's friendly tangents (#1706)
• WeightInitializers CUDAExt errors (#1709)
• adjust/adjust! does not work with ReactantOptimiser(OptimiserChain(...) (#1712)
• Tutorials button in doc homepage may linked to wrong url (#1717)
• Reproducible Seeds in Lux (#1723)