Published July 26, 2026
| Version v1
Technical note
Open
Long-Context Training and Positional Extrapolation: A Technical Note
Description
A technical note on long-context training and positional extrapolation for large language models, covering techniques such as rotary position embedding scaling and attention modifications used to extend context windows beyond their original training length.
Files
07_long-context-training-extrapolation.pdf
Files
(88.9 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:3e1941a3efbc3f45bd6e1cbe41f67ab6
|
88.9 kB | Preview Download |