Deepseek Presents Model with Scattered Attention to Reduce Inference Costs by Half

WWWhat's new

Factuality

Ownership

The Chinese company DeepSeek has launched an experimental model called V3.2-exp, aimed at optimizing the performance of operations in long contexts, one of the great technical challenges in the current language models. This version introduces an innovative system called DeepSeek Sparse Attention, whose aim is to minimize the computational load and thereby significantly reduce inference costs. When we talk about inference, we refer to the process…