Spectrans v0.1.0

This is the first public release of Spectrans, a modular library for spectral transformer implementations in PyTorch. This library provides alternatives to traditional attention mechanisms using spectral transforms.

🎯 Key Features

Core Implementations

• Fourier Neural Operator (FNO): Operator learning in Fourier space
• FNet: Pure Fourier transform-based token mixing
• Global Filter Network (GFNet): Learnable filters in frequency domain
• Adaptive Fourier Neural Operator (AFNO): Adaptive frequency filtering
• Wavelet Transformers: Multi-resolution analysis with wavelets
• Linear Spectral Transform (LST): Linear complexity attention alternative
• Spectral Cross-Attention: Efficient cross-attention in spectral domain

Transform Support

• Fast Fourier Transform (FFT): 1D and 2D implementations
• Discrete Cosine Transform (DCT)
• Discrete Sine Transform (DST)
• Discrete Wavelet Transform (DWT): 1D and 2D with multiple wavelet families
• Hadamard Transform
• Short-Time Fourier Transform (STFT)

Features

• 🔧 Modular Architecture: Mix and match components with registry system
• 📐 Mathematical Rigor: Proper complex number handling and gradient flow
• ⚡ Linearithmic Complexity: O(n log n) alternatives to O(n²) attention
• 🎛️ Configuration System: YAML-based configuration for experiments
• 🧪 Comprehensive Testing: Full test coverage with gradient checks
• 📚 Documentation: Detailed API docs and examples

📋 Requirements

• Python ≥ 3.13
• PyTorch ≥ 2.5.0
• NumPy ≥ 2.0.0
• See pyproject.toml for complete dependencies

🚀 Installation

pip install spectrans

💻 Quick Start

import torch
from spectrans.models import FNet

# Create an FNet model
model = FNet(
    vocab_size=30000,
    hidden_dim=768,
    num_layers=12,
    max_sequence_length=512,
    num_classes=2
)

# Or use specific components directly
from spectrans.layers.mixing import FourierMixing
mixing_layer = FourierMixing(hidden_dim=768)

# Forward pass
input_ids = torch.randint(0, 30000, (2, 128))
output = model(input_ids=input_ids)

## 📖 Documentation

Comprehensive documentation is available on [read the docs](https://spectrans.readthedocs.io).

## 🤝 Contributing

We welcome contributions! Please see our [Contributing Guide](https://github.com/aaronstevenwhite/spectrans/blob/main/CONTRIBUTING.md) for details.

## 📄 License

MIT License. See [LICENSE](https://github.com/aaronstevenwhite/spectrans/blob/main/LICENSE) file for details.