RecSSD: Near Data Processing for Solid State Drive Based Recommendation Inference

Neural personalized recommendation models are used across a wide variety of datacenter applications including search, social media, and entertainment. State-of-the-art models comprise large embedding tables that have billions of parameters requiring large memory capacities. Unfortunately, large and fast DRAM-based memories levy high infrastructure costs. While SSD-based storage solutions have up to an order of magnitude larger capacity, they exhibit low read latency and bandwidth degrading inference performance. In this paper we propose RecSSD, a near data processing based SSD memory system customized for neural recommendation inference. We demonstrate that RecSSD, reduces end-to-end model inference latency by 2×compared to off-the-shelf SSD-memories across eight industry-representative neural recommendation models.