Published July 17, 2026
| Version 1.0
Report
Open
Athenas-Symbiote-9B: A Parameter-Efficient Fine-Tuned LLM for Brazilian Portuguese Legal and Academic Domains
Description
Athenas-Symbiote-9B is an open-weights large language model optimized for Brazilian Portuguese through parameter-efficient fine-tuning (PEFT). Built on the Qwen3.5-9B-Base architecture, the model employs LoRA adapters with rsLoRA scaling, trained on constrained-compute hardware (1× NVIDIA T4, 16 GB VRAM, via Kaggle) using 4-bit quantization and the Unsloth optimization framework. Despite this extreme compute restriction, the model achieves 63.23% accuracy on the Brazilian Bar Examination (OAB) and 80.90% on the National High School Exam (ENEM), both evaluated in a 3-shot configuration using the EleutherAI lm-evaluation-harness on H100 infrastructure.
Files
Athenas-Symbiote-9B_Technical_Report.pdf
Files
(143.5 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:227827daaa7a921876d7fda71af252a3
|
143.5 kB | Preview Download |
Additional details
Software
- Repository URL
- https://huggingface.co/Kodjaoglanian/Athenas-Symbiote-9B
- Programming language
- Python
References
- Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. "LoRA: Low-Rank Adaptation of Large Language Models." ICLR, 2022.
- Kalajdzievski, D. "A Rank Stabilization Scaling Factor for Fine-Tuning LLMs with LoRA." arXiv:2312.03732, 2023.
- Gao, L., Tow, J., Abbasi, B., et al. "A Framework for Few-Shot Language Model Evaluation." EleutherAI, 2023. https://github.com/EleutherAI/lm-evaluation-harness
- Han, D. and Park, J. "Unsloth: Fine-Tune LLMs 2x Faster with 70% Less Memory." 2024. https://github.com/unslothai/unsloth
- von Werra, L. et al. "TRL: Transformer Reinforcement Learning." 2023. https://github.com/huggingface/trl
- Dettmers, T. "bitsandbytes: 8-bit and 4-bit Quantization for LLMs." 2023. https://github.com/TimDettmers/bitsandbytes
- Qwen Team. "Qwen3.5 Technical Report." 2026. https://huggingface.co/Qwen/Qwen3.5-9B
- ClassiCC-Corpus. "ClassiCC-PT: A Quality-Scored Portuguese Text Corpus." Hugging Face Datasets, 2024. https://huggingface.co/datasets/ClassiCC-Corpus/ClassiCC-PT
- Almeida, T. and Fidalgo, R. "OAB-Bench: Brazilian Bar Examination Benchmark." Maritaca AI, 2023. https://huggingface.co/datasets/maritaca-ai/oab-bench
- Almeida, T. and Fidalgo, R. "ENEM Challenge: National High School Exam Benchmark." Maritaca AI, 2023. https://huggingface.co/datasets/maritaca-ai/enem
- Liu, S. Y., Wang, C., Shi, H., and Tang, J. "PEFT: Parameter-Efficient Fine-Tuning." 2023. https://github.com/huggingface/peft
- Laitz, T., Almeida, T. S., Abonizio, H., Malaquias Junior, R., Bonás, G. K., Piau, M., Larcher, C., Pires, R., and Nogueira, R. "Sabiá-4 Technical Report." arXiv:2603.10213, February 2026.