Lux: Explicit Parameterization of Deep Neural Networks in Julia

WeightInitializers WeightInitializers-v1.1.4

Diff since WeightInitializers-v1.1.3

Merged pull requests:

• perf: benchmarking our models against Jax (Flax) (#1000) (@avik-pal)
• fix: use ignore derivatives for Reactant (#1342) (@avik-pal)
• ci: taming down ci timings (#1343) (@avik-pal)
• fix: remove onehotarrays patch (#1344) (@avik-pal)
• fix: bump reactant min version (#1345) (@avik-pal)
• CompatHelper: bump compat for MKL in [weakdeps] to 0.9 for package LuxLib, (keep existing compat) (#1346) (@github-actions[bot])
• CompatHelper: bump compat for MKL to 0.9 for package test, (keep existing compat) (#1347) (@github-actions[bot])
• chore: bump crate-ci/typos from 1.32.0 to 1.33.1 (#1349) (@dependabot[bot])
• chore: use uv for python (#1350) (@avik-pal)
• CompatHelper: bump compat for CairoMakie to 0.14 for package DDIM, (keep existing compat) (#1351) (@github-actions[bot])
• CompatHelper: bump compat for CairoMakie to 0.14 for package GravitationalWaveForm, (keep existing compat) (#1352) (@github-actions[bot])
• CompatHelper: bump compat for CairoMakie to 0.14 for package LSTMEncoderDecoder, (keep existing compat) (#1353) (@github-actions[bot])
• CompatHelper: bump compat for CairoMakie to 0.14 for package OptimizationIntegration, (keep existing compat) (#1354) (@github-actions[bot])
• CompatHelper: bump compat for CairoMakie to 0.14 for package PINN2DPDE, (keep existing compat) (#1355) (@github-actions[bot])
• CompatHelper: bump compat for CairoMakie to 0.14 for package PolynomialFitting, (keep existing compat) (#1356) (@github-actions[bot])
• CompatHelper: bump compat for CairoMakie to 0.14 for package RealNVP, (keep existing compat) (#1357) (@github-actions[bot])
• fix: remove type piracy (#1360) (@avik-pal)
• CompatHelper: bump compat for CairoMakie to 0.15 for package DDIM, (keep existing compat) (#1362) (@github-actions[bot])
• CompatHelper: bump compat for CairoMakie to 0.15 for package GravitationalWaveForm, (keep existing compat) (#1363) (@github-actions[bot])
• CompatHelper: bump compat for CairoMakie to 0.15 for package LSTMEncoderDecoder, (keep existing compat) (#1364) (@github-actions[bot])
• CompatHelper: bump compat for CairoMakie to 0.15 for package OptimizationIntegration, (keep existing compat) (#1365) (@github-actions[bot])
• CompatHelper: bump compat for CairoMakie to 0.15 for package PINN2DPDE, (keep existing compat) (#1366) (@github-actions[bot])
• CompatHelper: bump compat for CairoMakie to 0.15 for package PolynomialFitting, (keep existing compat) (#1367) (@github-actions[bot])
• CompatHelper: bump compat for CairoMakie to 0.15 for package RealNVP, (keep existing compat) (#1368) (@github-actions[bot])
• fix: use Int32 for GCN cora example (#1369) (@avik-pal)
• Fix backticks in examples/Basics (#1370) (@abhro)
• Fix up minor docs/docstrings formatting (#1371) (@abhro)
• feat: forwarddiff support for gather/scatter (#1373) (@avik-pal)
• fix: handle multi-device reactant (#1374) (@avik-pal)
• feat: serialization to tensorflow saved model (#1375) (@avik-pal)
• chore: update version for release (#1376) (@avik-pal)
• CompatHelper: add new compat entry for PythonCall at version 0.9 for package test, (keep existing compat) (#1377) (@github-actions[bot])
• fix: missing variable (#1379) (@avik-pal)
• chore: bump crate-ci/typos from 1.33.1 to 1.34.0 (#1382) (@dependabot[bot])
• fix: State returned by MultiHeadAttention is incompatible with the initialized state (#1384) (@yeruoforever)
• feat: annotate important parts of training loop (#1385) (@avik-pal)
• fix: bypass fused kernels for mooncake (for now) (#1387) (@avik-pal)
• feat: AutoMooncake for training lux models (#1388) (@avik-pal)
• CompatHelper: add new compat entry for Mooncake at version 0.4 for package test, (keep existing compat) (#1389) (@github-actions[bot])
• docs: cleanup CIFAR 10 example dependencies (#1391) (@avik-pal)
• CompatHelper: bump compat for AMDGPU in [weakdeps] to 2 for package LuxLib, (keep existing compat) (#1394) (@github-actions[bot])
• CompatHelper: bump compat for AMDGPU in [weakdeps] to 2 for package MLDataDevices, (keep existing compat) (#1395) (@github-actions[bot])
• CompatHelper: bump compat for AMDGPU in [weakdeps] to 2 for package WeightInitializers, (keep existing compat) (#1396) (@github-actions[bot])
• De-fragment markdown list in distributed_utils.md (#1397) (@abhro)
• Precompile environment before running tutorials (#1398) (@abhro)
• Add meaning of tuple elements in docs/tutorials.jl (#1399) (@abhro)

Closed issues:

• Export trained model for Tensorflow/PyTorch/C++? (#453)
• Externalize gradient computations to DifferentiationInterface.jl? (#544)
• Add a AutoMooncake dispatch for training Lux models (#1238)
• Don't materialize OneHotArrays with ReactantDevice (#1326)
• LSTMEncoderDecoder example broken (#1337)
• Error when freezing part of a model + Reactant (#1348)
• TrainState with mutliple Reactant Devices (#1358)
• ComponentArrays.jl type piracy? (#1359)
• Reactant.jl pass pipeline broke GCN Cora (#1361)
• jacobian_vector_product for Embedding (#1372)
• UndefVarError (:fname, LuxReactantExt) (#1378)
• Unable to evaluate a Lux model at a specific parameter set. (#1380)
• State Tuple returned by MultiHeadAttention is incompatible with the initialized tuple (#1383)
• MLIR Operand #1 doesn't dominate usage: Reactant + Cifar10 Example (#1386)