MSA: Memory Sparse Attention for Efficient End-to-End Memory Model Scaling to 100M Tokens

Authors/Creators

• Chen, Yu^{1, 2}
• Chen, Runkai^{1, 2, 3}
• Yi, Sheng^{1, 2}
• Zhao, Xinda^{1, 2}
• Li, Xiaohong^{1, 2}
• Zhang, Jianjin^{1, 2}
• Sun, Jun³
• Hu, Chuanrui^{1, 2}
• Han, Yunyun^{1, 2}
• Bing, Lidong²
• Deng, Yafeng^{1, 2}
• Chen, Tianqiao²