Lux: Explicit Parameterization of Deep Neural Networks in Julia

MLDataDevices MLDataDevices-v1.9.2

Diff since MLDataDevices-v1.9.1

Merged pull requests:

• fix: remove Optimisers.jl patch (#1247) (@avik-pal)
• chore: bump crate-ci/typos from 1.30.0 to 1.30.1 (#1258) (@dependabot[bot])
• Docs: Update autodiff.md to correct typos (#1259) (@sinhtrung)
• Update OptimizationIntegration/main.jl comment to match link title (#1261) (@sinhtrung)
• test: mark a test as broken (#1263) (@avik-pal)
• refactor: cleanup dependencies (Hwloc + CpuId) (#1264) (@avik-pal)
• docs: update readme (#1265) (@avik-pal)
• feat: add MultiHeadAttention (#1266) (@avik-pal)
• docs: use documenter citations (#1267) (@avik-pal)
• refactor: revert runic formatting (#1268) (@avik-pal)
• chore: bump crate-ci/typos from 1.30.1 to 1.30.2 (#1270) (@dependabot[bot])
• chore: remove unused imports (#1271) (@avik-pal)
• docs: refer to the Turing docs for BayesianNN (#1272) (@github-actions[bot])
• feat: support for ForwardDiff training (#1273) (@hstrey)
• fix: reactant typo (#1275) (@avik-pal)
• fix: outputsize of Dense (#1276) (@avik-pal)
• chore: bump forwarddiff to 1.0 (#1277) (@avik-pal)
• test: fix tests (#1278) (@avik-pal)
• fix: try fixing broken tests (#1279) (@avik-pal)
• show debug instead of warning if cant cuBLAS mult (#1280) (@ExpandingMan)
• chore: bump crate-ci/typos from 1.30.2 to 1.31.0 (#1281) (@dependabot[bot])
• chore: remove debug functionalities of reactant (#1285) (@avik-pal)
• fix: add a ReactantOptimisers wrapper (#1288) (@avik-pal)
• chore: bump crate-ci/typos from 1.31.0 to 1.31.1 (#1297) (@dependabot[bot])
• ci: use JuliaFormatter v1 (#1299) (@avik-pal)
• ci: multiple ci fixes (#1301) (@avik-pal)
• CompatHelper: bump compat for JET to 0.10 for package LuxTestUtils, (keep existing compat) (#1302) (@github-actions[bot])
• fix: new reactant version (#1303) (@avik-pal)
• fix: update Reactant training (#1304) (@avik-pal)
• fix: chain rules for recurrence tuple inputs (#1306) (@avik-pal)
• feat: fix return (#1307) (@avik-pal)
• fix: try increasing the samples in CI (#1309) (@avik-pal)
• fix: restrict dispatch types for cublaslt (#1311) (@avik-pal)
• chore: bump julia-actions/julia-format from 3 to 4 (#1313) (@dependabot[bot])
• feat: use 3rd order derivatives using Reactant (#1315) (@avik-pal)
• allow SelectDim to take arbitrary views (#1318) (@ExpandingMan)
• chore: bump crate-ci/typos from 1.31.1 to 1.32.0 (#1320) (@dependabot[bot])
• docs: fix wrong function names in RNG admonition in interface.md (#1325) (@KristianHolme)
• CompatHelper: bump compat for DocumenterVitepress to 0.2 for package docs, (keep existing compat) (#1328) (@github-actions[bot])
• CompatHelper: bump compat for Interpolations to 0.16 for package CIFAR10, (keep existing compat) (#1329) (@github-actions[bot])
• docs: lstm encoder decoder using Reactant (#1331) (@avik-pal)
• feat: lower embedding to direct indexing (#1332) (@avik-pal)
• fix: indexing (#1333) (@avik-pal)
• fix: reactant gradients + precision config (#1334) (@avik-pal)
• feat: emit batchnorm ops (#1336) (@avik-pal)

Closed issues:

• Per-Layer Profiling (#864)
• Integration of oneDNN for CPU operations (#1013)
• Optimisers.jl patch for Reactant Support (#1146)
• Emit Batchnorm Op for Training (#1208)
• Example from Training Lux Models using Optimization.jl not working in CPU (#1260)
• outputsize(::Dense) (#1262)
• Convolutional VAE for MNIST using Reactant failed to produce right results (#1274)
• GPU Inefficiency in Gradient Computation with Custom Recurrence (#1284)
• PINN2DPDE broke in the latest Optimisers Patch removal (#1286)
• Duplicating Scalars for Optimisers prevents CSE (#1289)
• Reactant 0.2.61 produces incorrect gradients (#1292)
• missing or incorrect ProjectTo method breaks Recurrence with Zygote (#1305)
• dense operations fail on views on nvidia due to missing method (#1308)
• Gradient of while loop with Reactant seems broken (#1316)
• getting concatenating and splitting working with Reactant/Enzyme (#1317)
• LuxLib doing LinearAlgebra.mul! on non-arrays can cause fallback to errors (#1322)
• gpu_device(device_id) attaches to first/zeroth device regardless of device_id (#1330)